Data Fetching
Data is essential for any UI Application and these applications are a bridge between users and the underlying data source(s), making it possible for users to interact with data in a meaningful way.
To manage data, refine needs a data provider
, which is a function that implements the DataProvider
interface. It is responsible for communicating with your API and making data available to refine applications. While you can use one of our built-in data providers, you can also easily create your own data provider matching your API.
refine passes relevant parameters like resource
name, or the id
of the record to your data provider, so data provider can make API calls to appropriate endpoints.
Once you provide data provider
to refine, you can utilize our data hooks (useOne
, useList
, useUpdate
) to easily manage your data from various sources, including REST, GraphQL, RPC, and SOAP.
Moreover, refine offers support for multiple data providers, allowing you to use different data providers for different resources. For instance, you can use REST for the posts
endpoint and GraphQL for the users
query.
Fetching Data
Imagine we want to fetch a record with the ID 123
from the products
endpoint. For this, we will use the useOne
hook. Under the hood, it calls the dataProvider.getOne
method from your data provider.
Dependencies: @refinedev/core@latest
Code Files
Updating Data
Now, let's update the record with the ID 124
from products
endpoint. To do this, we can use useUpdate
hook, which calls dataProvider.update
method under the hood.
In this example, we are updating product's price with a random value.
Dependencies: @refinedev/core@latest
Code Files
refine offers various data hooks for CRUD operations, you can see the list of these hooks below:
Hook | Method | Description |
---|---|---|
useOne | getOne | get a single record. |
useUpdate | update | update an existing record. |
useCreate | create | create a new record. |
useDelete | deleteOne | delete a single record. |
useList or useInfiniteList | getList | get a list of records. |
useApiUrl | getApiUrl | get the API URL. |
useCustom | custom | making custom API requests. |
useMany | getMany | get multiple records. |
useCreateMany | createMany | create multiple records. |
useDeleteMany | deleteMany | delete multiple records. |
useUpdateMany | updateMany | update multiple records. |
How refine treats data and state?
Data hooks uses TanStack Query under the hood. It takes care of managing the state for you. It provides data
, isLoading
, and error
states to help you handle loading, success, and error scenarios gracefully.
refine treats data and state in a structured and efficient manner, providing developers with powerful tools to manage data seamlessly within their applications. Here are some key aspects of how refine treats data and state:
Resource-Based Approach: Organizes data around resources, which are essentially models representing different data entities or API endpoints. These resources help structure your application's data management.
Invalidation: Automatically invalidates data after a successful mutation (e.g., creating, updating, or deleting a resource), ensuring that the UI is updated with the latest data.
Caching: Caches data to improve performance and deduplicates API calls.
Optimistic Updates: Supports optimistic updates, which means it will update the UI optimistically before the actual API call is complete. This enhances the user experience by reducing perceived latency.
Hooks for CRUD Operations: Offers a collection of hooks that align with common data operations like listing, creating, updating, and deleting data (
useList
,useCreate
,useUpdate
,useDelete
). In addition to these basic hooks, refine provides advanced hooks that are a composition of these fundamental ones for handling more complex tasks (useForm
,useTable
,useSelect
).Integration with UI Libraries: Works seamlessly with popular UI libraries. It provides a structured approach to represent data within these libraries.
Realtime Updates: Allowing your application to reflect changes in data as they occur.
Meta usage
meta
is a special property that can be used to pass additional information to your data provider methods through data hooks like useOne
, useList
, useForm
from anywhere accros your application.
The capabilities of meta
properties depend on your data provider's implementation. While some may use additional features through meta
, others may not use them or follow a different approach.
Here are some examples of meta
usage:
- Passing additional headers or parameters to the request.
- Generate GraphQL queries.
- Multi-tenancy support (passing the tenant id to the request).
In the example below, we are passing meta.foo
property to the useOne
hook. Then, we are using this property to pass additional headers to the request.
import { DataProvider, useOne } from "@refinedev/core";
useOne({
resource: "products",
id: 1,
meta: {
foo: "bar",
},
});
export const dataProvider = (apiUrl: string): DataProvider => ({
getOne: async ({ resource, id, meta }) => {
const response = await fetch(`${apiUrl}/${resource}/${id}`, {
headers: {
"x-foo": meta.foo,
},
});
const data = await response.json();
return {
data,
};
},
...
});
GraphQL
As mentioned above, meta
property can also be used to generate GraphQL queries.
meta.fields
, meta.variables
, and meta.operation
fields implements gql-query-builder interface, so this interface can be used to easily generate GraphQL queries.
import { DataProvider, useOne } from "@refinedev/core";
import * as gql from "gql-query-builder";
import { GraphQLClient } from "graphql-request";
useOne({
resource: "products",
id: 1,
meta: {
fields: [
"id",
"title",
{
category: ["title"],
},
],
},
});
const dataProvider = (client: GraphQLClient): DataProvider => {
getOne: async ({ resource, id, meta }) => {
const operation = meta?.operation || resource;
const { query, variables } = gql.query({
operation,
variables: {
id: { value: id, type: "ID", required: true },
},
fields: meta?.fields,
variables: meta?.variables,
});
console.log(query);
// "query ($id: ID!) { products (id: $id) { id, title, category { title } } }"
const response = await client.request(query, variables);
return {
data: response[resource],
};
};
...
};
Also, you can check refine's built-in GraphQL data providers to handle communication with your GraphQL APIs or use them as a starting point.
Multiple Data Providers
Using multiple data providers in refine allows you to work with various APIs or data sources in a single application. You might use different data providers for different parts of your app.
Each data provider can have its own configuration, making it easier to manage complex data scenarios within a single application. This flexibility is handy when dealing with various data structures and APIs.
For example, we want to fetch:
products
fromhttps://api.finefoods.refine.dev
user
fromhttps://api.fake-rest.refine.dev
.
As you can see the example below:
- We are defining multiple data providers in
App.tsx
. - Using
dataProviderName
field to specify which data provider to use in data hooks inhome-page.tsx
.
Dependencies: @refinedev/core@latest
Code Files
Handling errors
refine expects errors to be extended from HttpError. We believe that having consistent error interface makes it easier to handle errors coming from your API.
When implemented correctly, refine offers several advantages in error handling:
- Notification: If you have
notificationProvider
, refine will automatically show a notification when an error occurs. - Server-Side Validation: Shows errors coming from the API on the corresponding form fields.
- Optimistic Updates: Instantly update UI when you send a mutation and automatically revert the changes if an error occurs during the mutation.
Dependencies: @refinedev/core@latest
Code Files
Listing Data
Imagine we need to fetch a list of records from the products
endpoint. For this, we can use useList
or useInfiniteList
hooks. It calls dataProvider.getList
method from your data provider, returns data
and total
fields from the response.
Dependencies: @refinedev/core@latest
Code Files
Filters, Sorters and Pagination
We fetched all the products from the products
endpoint in the previous example. But in real world, we usually need to fetch a subset of the data.
refine provides a unified filters
, sorters
, and pagination
parameters in data hooks to pass your data provider
methods, making it possible to fetch the data you need with any complexity. It's data provider's responsibility to handle these parameters and modify the request sent to your API.
Now let's make it more realistic example by adding filters, sorters, and pagination.
We want to:
- Fetch 5 products
- With
material
field equals towooden
- Sorted by
ID
field indescending
order
For this purpose, we can pass additional parameters to useList
hook like filters
, sorters
, and pagination
.
useList
calls the dataProvider.getList
method under the hood with the given parameters. We will use these parameters modify our request sent to our API.
Dependencies: @refinedev/core@latest
Code Files
While the example above is simple, it's also possible to build more complex queries with filters
and sorters
.
For instance, we can fetch products:
- With wooden material
- Belongs to category ID 45
- OR have a price between 1000 and 2000.
import { DataProvider, useList } from "@refinedev/core";
useList({
resource: "products",
pagination: {
current: 1,
pageSize: 10,
},
filters: [
{
operator: "and",
value: [
{ field: "material", operator: "eq", value: "wooden" },
{ field: "category.id", operator: "eq", value: 45 },
],
},
{
operator: "or",
value: [
{ field: "price", operator: "gte", value: 1000 },
{ field: "price", operator: "lte", value: 2000 },
],
},
],
});
Relationships
refine handles data relations with data hooks(eg: useOne
, useMany
, etc.). This compositional design allows you to flexibly and efficiently manage data relationships to suit your specific requirements.
One-to-One
In a one-to-one relationship, each thing matches with just one other thing. It's like a unique partnership.
For instance, a product can have only one product detail.
┌──────────────┐ ┌────────────────┐
│ Products │ │ ProductDetail │
│--------------│ │----------------│
│ id │───────│ id │
│ name │ │ weight │
│ price │ │ dimensions │
│ description │ │ productId │
│ detail │ │ │
│ │ │ │
└──────────────┘ └────────────────┘
We can use the useOne
hook to fetch the detail of a product.
Dependencies: @refinedev/core@latest
Code Files
One-to-Many
In a one-to-many relationship, each resource matches with many other resource. It's like a parent with many children.
For instance, a products can have many reviews.
┌──────────────┐ ┌────────────────┐
│ Products │ │ Reviews │
│--------------│ │----------------│
│ id │───┐ │ id │
│ name │ │ │ rating │
│ price │ │ │ comment │
│ description │ │ │ user │
│ detail │ └───│ product │
│ │ │ │
└──────────────┘ └────────────────┘
We can use the useList
hook and filter by the product ID to fetch the reviews of a product.
Dependencies: @refinedev/core@latest
Code Files
Many-to-Many
In a many-to-many relationship, each resource matches with many other resources, and each of those resources matches with many other resources.
For instance, products can have many categories, and categories can have many products.
┌──────────────┐ ┌───────────────────┐ ┌──────────────┐
│ Products │ │ ProductCategories │ │ Categories │
│--------------│ │----------------───│ │--------------│
│ id │───┐ │ id │ ┌───│ id │
│ name │ └───│ productId │ │ │ name │
│ price │ │ categoryId │───┘ │ description │
│ description │ │ │ │ │
│ detail │ │ │ │ │
│ │ │ │ │ │
└──────────────┘ └───────────────────┘ └──────────────┘
In this case, we can use the useMany
hook to fetch the categories of a product and the useMany
hook to fetch the products of a category.
import { DataProvider, useMany } from "@refinedev/core";
const { data: productCategories } = useList({
resource: "productCategories",
});
const { data: products } = useMany({
resource: "products",
ids: productCategories.map((productCategory) => productCategory.productId),
queryOptions: {
enabled: productCategories.length > 0,
},
});
const { data: categories } = useMany({
resource: "categories",
ids: productCategories.map((productCategory) => productCategory.categoryId),
queryOptions: {
enabled: productCategories.length > 0,
},
});
Authentication
Imagine you want to fetch a data from a protected API. To do this, you will first need to obtain your authentication token and you will need to send this token with every request.
In refine we handle authentication with Auth Provider. To get token from the API, we will use the authProvider.login
method. Then, we will use <Authenticated />
component to to render the appropriate components.
After obtaining the token, we'll use Axios interceptors to include the token in the headers of all requests.
Dependencies: @refinedev/core@latest,axios@^0.26.1
Code Files
TanStack Query QueryClient
To modify the QueryClient
instance, you can use the reactQuery
prop of the <Refine />
component.
dataProvider
interface
To better understand the data provider interface, we have created an example that demonstrates how the required methods are implemented. For more comprehensive and diverse examples, you can refer to the supported data providers section.
In this example, we implemented data provider to support JSON placeholder API.
import {
DataProvider,
HttpError,
Pagination,
CrudSorting,
CrudFilters,
CrudOperators,
} from "@refinedev/core";
import { stringify } from "query-string";
import axios, { AxiosInstance } from "axios";
type MethodTypes = "get" | "delete" | "head" | "options";
type MethodTypesWithBody = "post" | "put" | "patch";
const axiosInstance = axios.create();
export const dataProvider = (
apiUrl: string,
// get axios instance from user or use default one.
httpClient: AxiosInstance = axiosInstance,
): DataProvider => ({
getOne: async ({ resource, id, meta }) => {
const url = `${apiUrl}/${resource}/${id}`;
const { headers, method } = meta ?? {};
const requestMethod = (method as MethodTypes) ?? "get";
const { data } = await httpClient[requestMethod](url, { headers });
return {
data,
};
},
update: async ({ resource, id, variables, meta }) => {
const url = `${apiUrl}/${resource}/${id}`;
const { headers, method } = meta ?? {};
const requestMethod = (method as MethodTypesWithBody) ?? "patch";
const { data } = await httpClient[requestMethod](url, variables, {
headers,
});
return {
data,
};
},
create: async ({ resource, variables, meta }) => {
const url = `${apiUrl}/${resource}`;
const { headers, method } = meta ?? {};
const requestMethod = (method as MethodTypesWithBody) ?? "post";
const { data } = await httpClient[requestMethod](url, variables, {
headers,
});
return {
data,
};
},
deleteOne: async ({ resource, id, variables, meta }) => {
const url = `${apiUrl}/${resource}/${id}`;
const { headers, method } = meta ?? {};
const requestMethod = (method as MethodTypesWithBody) ?? "delete";
const { data } = await httpClient[requestMethod](url, {
data: variables,
headers,
});
return {
data,
};
},
getList: async ({ resource, pagination, sorters, filters, meta }) => {
const url = `${apiUrl}/${resource}`;
const { headers: headersFromMeta, method } = meta ?? {};
const requestMethod = (method as MethodTypes) ?? "get";
// init query object for pagination and sorting
const query: {
_start?: number;
_end?: number;
_sort?: string;
_order?: string;
} = {};
const generatedPagination = generatePagination(pagination);
if (generatedPagination) {
const { _start, _end } = generatedPagination;
query._start = _start;
query._end = _end;
}
const generatedSort = generateSort(sorters);
if (generatedSort) {
const { _sort, _order } = generatedSort;
query._sort = _sort.join(",");
query._order = _order.join(",");
}
const queryFilters = generateFilter(filters);
const { data, headers } = await httpClient[requestMethod](
`${url}?${stringify(query)}&${stringify(queryFilters)}`,
{
headers: headersFromMeta,
},
);
const total = +headers["x-total-count"];
return {
data,
total: total || data.length,
};
},
getApiUrl: () => apiUrl,
});
// Convert axios errors to HttpError on every response.
axiosInstance.interceptors.response.use(
(response) => {
return response;
},
(error) => {
const customError: HttpError = {
...error,
message: error.response?.data?.message,
statusCode: error.response?.status,
};
return Promise.reject(customError);
},
);
// convert refine CrudOperators to the format that API accepts.
const mapOperator = (operator: CrudOperators): string => {
switch (operator) {
case "ne":
case "gte":
case "lte":
return `_${operator}`;
case "contains":
return "_like";
case "eq":
default:
return "";
}
};
// generate query string from refine CrudFilters to the format that API accepts.
const generateFilter = (filters?: CrudFilters) => {
const queryFilters: { [key: string]: string } = {};
if (filters) {
filters.map((filter) => {
if (filter.operator === "or" || filter.operator === "and") {
throw new Error(
`[@refinedev/simple-rest]: \`operator: ${filter.operator}\` is not supported. You can create custom data provider. https://refine.dev/docs/api-reference/core/providers/data-provider/#creating-a-data-provider`,
);
}
if ("field" in filter) {
const { field, operator, value } = filter;
if (field === "q") {
queryFilters[field] = value;
return;
}
const mappedOperator = mapOperator(operator);
queryFilters[`${field}${mappedOperator}`] = value;
}
});
}
return queryFilters;
};
// generate query string from refine CrudSorting to the format that API accepts.
const generateSort = (sorters?: CrudSorting) => {
if (sorters && sorters.length > 0) {
const _sort: string[] = [];
const _order: string[] = [];
sorters.map((item) => {
_sort.push(item.field);
_order.push(item.order);
});
return {
_sort,
_order,
};
}
return;
};
// generate query string from refine Pagination to the format that API accepts.
const generatePagination = (pagination?: Pagination) => {
// pagination is optional on data hooks, so we need to set default values.
const { current = 1, pageSize = 10, mode = "server" } = pagination ?? {};
const query: {
_start?: number;
_end?: number;
} = {};
if (mode === "server") {
query._start = (current - 1) * pageSize;
query._end = current * pageSize;
}
return query;
};
To learn more about the dataProvider
interface, check out the reference page.
Supported data providers
refine supports many data providers. To include them in your project, you can use npm install [packageName]
or you can select the preferred data provider with the npm create refine-app@latest projectName
during the project creation phase with CLI. This will allow you to easily use these data providers in your project.
- Simple REST API
- GraphQL
- NestJS CRUD
- Nestjs-Query
- Airtable
- Strapi - Strapi v4
- Strapi GraphQL
- Supabase
- Hasura
- Appwrite
- Medusa
Community ❤️
- Firebase by rturan29
- Directus by tspvivek
- Elide by chirdeeptomar
- Elide GraphQL by chirdeeptomar
- useGenerated by usegen
- Hygraph by acomagu
- Sanity by hirenf14
- SQLite by mateusabelli
- JSON:API by mahirmahdi
If you have created a custom data provider and would like to share it with the community, please don't hesitate to get in touch with us. We would be happy to include it on this page for others to use.
Data hooks
Hook | Method | Description |
---|---|---|
useOne | getOne | get a single record. |
useUpdate | update | update an existing record. |
useCreate | create | create a new record. |
useDelete | deleteOne | delete a single record. |
useList or useInfiniteList | getList | get a list of records. |
useApiUrl | getApiUrl | get the API URL. |
useCustom | custom | making custom API requests. |
useMany | getMany | get multiple records. |
useCreateMany | createMany | create multiple records. |
useDeleteMany | deleteMany | delete multiple records. |
useUpdateMany | updateMany | update multiple records. |
- Fetching Data
- Updating Data
- How refine treats data and state?
- Meta usage
- GraphQL
- Multiple Data Providers
- Handling errors
- Listing Data
- Filters, Sorters and Pagination
- Relationships
- One-to-One
- One-to-Many
- Many-to-Many
- Authentication
- TanStack Query
QueryClient
dataProvider
interface- Supported data providers
- Data hooks