About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / filesystems / caching / backend-api.txt




Custom Search

Based on kernel version 3.15.4. Page generated on 2014-07-07 09:02 EST.

1				  ==========================
2				  FS-CACHE CACHE BACKEND API
3				  ==========================
4	
5	The FS-Cache system provides an API by which actual caches can be supplied to
6	FS-Cache for it to then serve out to network filesystems and other interested
7	parties.
8	
9	This API is declared in <linux/fscache-cache.h>.
10	
11	
12	====================================
13	INITIALISING AND REGISTERING A CACHE
14	====================================
15	
16	To start off, a cache definition must be initialised and registered for each
17	cache the backend wants to make available.  For instance, CacheFS does this in
18	the fill_super() operation on mounting.
19	
20	The cache definition (struct fscache_cache) should be initialised by calling:
21	
22		void fscache_init_cache(struct fscache_cache *cache,
23					struct fscache_cache_ops *ops,
24					const char *idfmt,
25					...);
26	
27	Where:
28	
29	 (*) "cache" is a pointer to the cache definition;
30	
31	 (*) "ops" is a pointer to the table of operations that the backend supports on
32	     this cache; and
33	
34	 (*) "idfmt" is a format and printf-style arguments for constructing a label
35	     for the cache.
36	
37	
38	The cache should then be registered with FS-Cache by passing a pointer to the
39	previously initialised cache definition to:
40	
41		int fscache_add_cache(struct fscache_cache *cache,
42				      struct fscache_object *fsdef,
43				      const char *tagname);
44	
45	Two extra arguments should also be supplied:
46	
47	 (*) "fsdef" which should point to the object representation for the FS-Cache
48	     master index in this cache.  Netfs primary index entries will be created
49	     here.  FS-Cache keeps the caller's reference to the index object if
50	     successful and will release it upon withdrawal of the cache.
51	
52	 (*) "tagname" which, if given, should be a text string naming this cache.  If
53	     this is NULL, the identifier will be used instead.  For CacheFS, the
54	     identifier is set to name the underlying block device and the tag can be
55	     supplied by mount.
56	
57	This function may return -ENOMEM if it ran out of memory or -EEXIST if the tag
58	is already in use.  0 will be returned on success.
59	
60	
61	=====================
62	UNREGISTERING A CACHE
63	=====================
64	
65	A cache can be withdrawn from the system by calling this function with a
66	pointer to the cache definition:
67	
68		void fscache_withdraw_cache(struct fscache_cache *cache);
69	
70	In CacheFS's case, this is called by put_super().
71	
72	
73	========
74	SECURITY
75	========
76	
77	The cache methods are executed one of two contexts:
78	
79	 (1) that of the userspace process that issued the netfs operation that caused
80	     the cache method to be invoked, or
81	
82	 (2) that of one of the processes in the FS-Cache thread pool.
83	
84	In either case, this may not be an appropriate context in which to access the
85	cache.
86	
87	The calling process's fsuid, fsgid and SELinux security identities may need to
88	be masqueraded for the duration of the cache driver's access to the cache.
89	This is left to the cache to handle; FS-Cache makes no effort in this regard.
90	
91	
92	===================================
93	CONTROL AND STATISTICS PRESENTATION
94	===================================
95	
96	The cache may present data to the outside world through FS-Cache's interfaces
97	in sysfs and procfs - the former for control and the latter for statistics.
98	
99	A sysfs directory called /sys/fs/fscache/<cachetag>/ is created if CONFIG_SYSFS
100	is enabled.  This is accessible through the kobject struct fscache_cache::kobj
101	and is for use by the cache as it sees fit.
102	
103	
104	========================
105	RELEVANT DATA STRUCTURES
106	========================
107	
108	 (*) Index/Data file FS-Cache representation cookie:
109	
110		struct fscache_cookie {
111			struct fscache_object_def	*def;
112			struct fscache_netfs		*netfs;
113			void				*netfs_data;
114			...
115		};
116	
117	     The fields that might be of use to the backend describe the object
118	     definition, the netfs definition and the netfs's data for this cookie.
119	     The object definition contain functions supplied by the netfs for loading
120	     and matching index entries; these are required to provide some of the
121	     cache operations.
122	
123	
124	 (*) In-cache object representation:
125	
126		struct fscache_object {
127			int				debug_id;
128			enum {
129				FSCACHE_OBJECT_RECYCLING,
130				...
131			}				state;
132			spinlock_t			lock
133			struct fscache_cache		*cache;
134			struct fscache_cookie		*cookie;
135			...
136		};
137	
138	     Structures of this type should be allocated by the cache backend and
139	     passed to FS-Cache when requested by the appropriate cache operation.  In
140	     the case of CacheFS, they're embedded in CacheFS's internal object
141	     structures.
142	
143	     The debug_id is a simple integer that can be used in debugging messages
144	     that refer to a particular object.  In such a case it should be printed
145	     using "OBJ%x" to be consistent with FS-Cache.
146	
147	     Each object contains a pointer to the cookie that represents the object it
148	     is backing.  An object should retired when put_object() is called if it is
149	     in state FSCACHE_OBJECT_RECYCLING.  The fscache_object struct should be
150	     initialised by calling fscache_object_init(object).
151	
152	
153	 (*) FS-Cache operation record:
154	
155		struct fscache_operation {
156			atomic_t		usage;
157			struct fscache_object	*object;
158			unsigned long		flags;
159		#define FSCACHE_OP_EXCLUSIVE
160			void (*processor)(struct fscache_operation *op);
161			void (*release)(struct fscache_operation *op);
162			...
163		};
164	
165	     FS-Cache has a pool of threads that it uses to give CPU time to the
166	     various asynchronous operations that need to be done as part of driving
167	     the cache.  These are represented by the above structure.  The processor
168	     method is called to give the op CPU time, and the release method to get
169	     rid of it when its usage count reaches 0.
170	
171	     An operation can be made exclusive upon an object by setting the
172	     appropriate flag before enqueuing it with fscache_enqueue_operation().  If
173	     an operation needs more processing time, it should be enqueued again.
174	
175	
176	 (*) FS-Cache retrieval operation record:
177	
178		struct fscache_retrieval {
179			struct fscache_operation op;
180			struct address_space	*mapping;
181			struct list_head	*to_do;
182			...
183		};
184	
185	     A structure of this type is allocated by FS-Cache to record retrieval and
186	     allocation requests made by the netfs.  This struct is then passed to the
187	     backend to do the operation.  The backend may get extra refs to it by
188	     calling fscache_get_retrieval() and refs may be discarded by calling
189	     fscache_put_retrieval().
190	
191	     A retrieval operation can be used by the backend to do retrieval work.  To
192	     do this, the retrieval->op.processor method pointer should be set
193	     appropriately by the backend and fscache_enqueue_retrieval() called to
194	     submit it to the thread pool.  CacheFiles, for example, uses this to queue
195	     page examination when it detects PG_lock being cleared.
196	
197	     The to_do field is an empty list available for the cache backend to use as
198	     it sees fit.
199	
200	
201	 (*) FS-Cache storage operation record:
202	
203		struct fscache_storage {
204			struct fscache_operation op;
205			pgoff_t			store_limit;
206			...
207		};
208	
209	     A structure of this type is allocated by FS-Cache to record outstanding
210	     writes to be made.  FS-Cache itself enqueues this operation and invokes
211	     the write_page() method on the object at appropriate times to effect
212	     storage.
213	
214	
215	================
216	CACHE OPERATIONS
217	================
218	
219	The cache backend provides FS-Cache with a table of operations that can be
220	performed on the denizens of the cache.  These are held in a structure of type:
221	
222		struct fscache_cache_ops
223	
224	 (*) Name of cache provider [mandatory]:
225	
226		const char *name
227	
228	     This isn't strictly an operation, but should be pointed at a string naming
229	     the backend.
230	
231	
232	 (*) Allocate a new object [mandatory]:
233	
234		struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
235						       struct fscache_cookie *cookie)
236	
237	     This method is used to allocate a cache object representation to back a
238	     cookie in a particular cache.  fscache_object_init() should be called on
239	     the object to initialise it prior to returning.
240	
241	     This function may also be used to parse the index key to be used for
242	     multiple lookup calls to turn it into a more convenient form.  FS-Cache
243	     will call the lookup_complete() method to allow the cache to release the
244	     form once lookup is complete or aborted.
245	
246	
247	 (*) Look up and create object [mandatory]:
248	
249		void (*lookup_object)(struct fscache_object *object)
250	
251	     This method is used to look up an object, given that the object is already
252	     allocated and attached to the cookie.  This should instantiate that object
253	     in the cache if it can.
254	
255	     The method should call fscache_object_lookup_negative() as soon as
256	     possible if it determines the object doesn't exist in the cache.  If the
257	     object is found to exist and the netfs indicates that it is valid then
258	     fscache_obtained_object() should be called once the object is in a
259	     position to have data stored in it.  Similarly, fscache_obtained_object()
260	     should also be called once a non-present object has been created.
261	
262	     If a lookup error occurs, fscache_object_lookup_error() should be called
263	     to abort the lookup of that object.
264	
265	
266	 (*) Release lookup data [mandatory]:
267	
268		void (*lookup_complete)(struct fscache_object *object)
269	
270	     This method is called to ask the cache to release any resources it was
271	     using to perform a lookup.
272	
273	
274	 (*) Increment object refcount [mandatory]:
275	
276		struct fscache_object *(*grab_object)(struct fscache_object *object)
277	
278	     This method is called to increment the reference count on an object.  It
279	     may fail (for instance if the cache is being withdrawn) by returning NULL.
280	     It should return the object pointer if successful.
281	
282	
283	 (*) Lock/Unlock object [mandatory]:
284	
285		void (*lock_object)(struct fscache_object *object)
286		void (*unlock_object)(struct fscache_object *object)
287	
288	     These methods are used to exclusively lock an object.  It must be possible
289	     to schedule with the lock held, so a spinlock isn't sufficient.
290	
291	
292	 (*) Pin/Unpin object [optional]:
293	
294		int (*pin_object)(struct fscache_object *object)
295		void (*unpin_object)(struct fscache_object *object)
296	
297	     These methods are used to pin an object into the cache.  Once pinned an
298	     object cannot be reclaimed to make space.  Return -ENOSPC if there's not
299	     enough space in the cache to permit this.
300	
301	
302	 (*) Check coherency state of an object [mandatory]:
303	
304		int (*check_consistency)(struct fscache_object *object)
305	
306	     This method is called to have the cache check the saved auxiliary data of
307	     the object against the netfs's idea of the state.  0 should be returned
308	     if they're consistent and -ESTALE otherwise.  -ENOMEM and -ERESTARTSYS
309	     may also be returned.
310	
311	 (*) Update object [mandatory]:
312	
313		int (*update_object)(struct fscache_object *object)
314	
315	     This is called to update the index entry for the specified object.  The
316	     new information should be in object->cookie->netfs_data.  This can be
317	     obtained by calling object->cookie->def->get_aux()/get_attr().
318	
319	
320	 (*) Invalidate data object [mandatory]:
321	
322		int (*invalidate_object)(struct fscache_operation *op)
323	
324	     This is called to invalidate a data object (as pointed to by op->object).
325	     All the data stored for this object should be discarded and an
326	     attr_changed operation should be performed.  The caller will follow up
327	     with an object update operation.
328	
329	     fscache_op_complete() must be called on op before returning.
330	
331	
332	 (*) Discard object [mandatory]:
333	
334		void (*drop_object)(struct fscache_object *object)
335	
336	     This method is called to indicate that an object has been unbound from its
337	     cookie, and that the cache should release the object's resources and
338	     retire it if it's in state FSCACHE_OBJECT_RECYCLING.
339	
340	     This method should not attempt to release any references held by the
341	     caller.  The caller will invoke the put_object() method as appropriate.
342	
343	
344	 (*) Release object reference [mandatory]:
345	
346		void (*put_object)(struct fscache_object *object)
347	
348	     This method is used to discard a reference to an object.  The object may
349	     be freed when all the references to it are released.
350	
351	
352	 (*) Synchronise a cache [mandatory]:
353	
354		void (*sync)(struct fscache_cache *cache)
355	
356	     This is called to ask the backend to synchronise a cache with its backing
357	     device.
358	
359	
360	 (*) Dissociate a cache [mandatory]:
361	
362		void (*dissociate_pages)(struct fscache_cache *cache)
363	
364	     This is called to ask a cache to perform any page dissociations as part of
365	     cache withdrawal.
366	
367	
368	 (*) Notification that the attributes on a netfs file changed [mandatory]:
369	
370		int (*attr_changed)(struct fscache_object *object);
371	
372	     This is called to indicate to the cache that certain attributes on a netfs
373	     file have changed (for example the maximum size a file may reach).  The
374	     cache can read these from the netfs by calling the cookie's get_attr()
375	     method.
376	
377	     The cache may use the file size information to reserve space on the cache.
378	     It should also call fscache_set_store_limit() to indicate to FS-Cache the
379	     highest byte it's willing to store for an object.
380	
381	     This method may return -ve if an error occurred or the cache object cannot
382	     be expanded.  In such a case, the object will be withdrawn from service.
383	
384	     This operation is run asynchronously from FS-Cache's thread pool, and
385	     storage and retrieval operations from the netfs are excluded during the
386	     execution of this operation.
387	
388	
389	 (*) Reserve cache space for an object's data [optional]:
390	
391		int (*reserve_space)(struct fscache_object *object, loff_t size);
392	
393	     This is called to request that cache space be reserved to hold the data
394	     for an object and the metadata used to track it.  Zero size should be
395	     taken as request to cancel a reservation.
396	
397	     This should return 0 if successful, -ENOSPC if there isn't enough space
398	     available, or -ENOMEM or -EIO on other errors.
399	
400	     The reservation may exceed the current size of the object, thus permitting
401	     future expansion.  If the amount of space consumed by an object would
402	     exceed the reservation, it's permitted to refuse requests to allocate
403	     pages, but not required.  An object may be pruned down to its reservation
404	     size if larger than that already.
405	
406	
407	 (*) Request page be read from cache [mandatory]:
408	
409		int (*read_or_alloc_page)(struct fscache_retrieval *op,
410					  struct page *page,
411					  gfp_t gfp)
412	
413	     This is called to attempt to read a netfs page from the cache, or to
414	     reserve a backing block if not.  FS-Cache will have done as much checking
415	     as it can before calling, but most of the work belongs to the backend.
416	
417	     If there's no page in the cache, then -ENODATA should be returned if the
418	     backend managed to reserve a backing block; -ENOBUFS or -ENOMEM if it
419	     didn't.
420	
421	     If there is suitable data in the cache, then a read operation should be
422	     queued and 0 returned.  When the read finishes, fscache_end_io() should be
423	     called.
424	
425	     The fscache_mark_pages_cached() should be called for the page if any cache
426	     metadata is retained.  This will indicate to the netfs that the page needs
427	     explicit uncaching.  This operation takes a pagevec, thus allowing several
428	     pages to be marked at once.
429	
430	     The retrieval record pointed to by op should be retained for each page
431	     queued and released when I/O on the page has been formally ended.
432	     fscache_get/put_retrieval() are available for this purpose.
433	
434	     The retrieval record may be used to get CPU time via the FS-Cache thread
435	     pool.  If this is desired, the op->op.processor should be set to point to
436	     the appropriate processing routine, and fscache_enqueue_retrieval() should
437	     be called at an appropriate point to request CPU time.  For instance, the
438	     retrieval routine could be enqueued upon the completion of a disk read.
439	     The to_do field in the retrieval record is provided to aid in this.
440	
441	     If an I/O error occurs, fscache_io_error() should be called and -ENOBUFS
442	     returned if possible or fscache_end_io() called with a suitable error
443	     code.
444	
445	     fscache_put_retrieval() should be called after a page or pages are dealt
446	     with.  This will complete the operation when all pages are dealt with.
447	
448	
449	 (*) Request pages be read from cache [mandatory]:
450	
451		int (*read_or_alloc_pages)(struct fscache_retrieval *op,
452					   struct list_head *pages,
453					   unsigned *nr_pages,
454					   gfp_t gfp)
455	
456	     This is like the read_or_alloc_page() method, except it is handed a list
457	     of pages instead of one page.  Any pages on which a read operation is
458	     started must be added to the page cache for the specified mapping and also
459	     to the LRU.  Such pages must also be removed from the pages list and
460	     *nr_pages decremented per page.
461	
462	     If there was an error such as -ENOMEM, then that should be returned; else
463	     if one or more pages couldn't be read or allocated, then -ENOBUFS should
464	     be returned; else if one or more pages couldn't be read, then -ENODATA
465	     should be returned.  If all the pages are dispatched then 0 should be
466	     returned.
467	
468	
469	 (*) Request page be allocated in the cache [mandatory]:
470	
471		int (*allocate_page)(struct fscache_retrieval *op,
472				     struct page *page,
473				     gfp_t gfp)
474	
475	     This is like the read_or_alloc_page() method, except that it shouldn't
476	     read from the cache, even if there's data there that could be retrieved.
477	     It should, however, set up any internal metadata required such that
478	     the write_page() method can write to the cache.
479	
480	     If there's no backing block available, then -ENOBUFS should be returned
481	     (or -ENOMEM if there were other problems).  If a block is successfully
482	     allocated, then the netfs page should be marked and 0 returned.
483	
484	
485	 (*) Request pages be allocated in the cache [mandatory]:
486	
487		int (*allocate_pages)(struct fscache_retrieval *op,
488				      struct list_head *pages,
489				      unsigned *nr_pages,
490				      gfp_t gfp)
491	
492	     This is an multiple page version of the allocate_page() method.  pages and
493	     nr_pages should be treated as for the read_or_alloc_pages() method.
494	
495	
496	 (*) Request page be written to cache [mandatory]:
497	
498		int (*write_page)(struct fscache_storage *op,
499				  struct page *page);
500	
501	     This is called to write from a page on which there was a previously
502	     successful read_or_alloc_page() call or similar.  FS-Cache filters out
503	     pages that don't have mappings.
504	
505	     This method is called asynchronously from the FS-Cache thread pool.  It is
506	     not required to actually store anything, provided -ENODATA is then
507	     returned to the next read of this page.
508	
509	     If an error occurred, then a negative error code should be returned,
510	     otherwise zero should be returned.  FS-Cache will take appropriate action
511	     in response to an error, such as withdrawing this object.
512	
513	     If this method returns success then FS-Cache will inform the netfs
514	     appropriately.
515	
516	
517	 (*) Discard retained per-page metadata [mandatory]:
518	
519		void (*uncache_page)(struct fscache_object *object, struct page *page)
520	
521	     This is called when a netfs page is being evicted from the pagecache.  The
522	     cache backend should tear down any internal representation or tracking it
523	     maintains for this page.
524	
525	
526	==================
527	FS-CACHE UTILITIES
528	==================
529	
530	FS-Cache provides some utilities that a cache backend may make use of:
531	
532	 (*) Note occurrence of an I/O error in a cache:
533	
534		void fscache_io_error(struct fscache_cache *cache)
535	
536	     This tells FS-Cache that an I/O error occurred in the cache.  After this
537	     has been called, only resource dissociation operations (object and page
538	     release) will be passed from the netfs to the cache backend for the
539	     specified cache.
540	
541	     This does not actually withdraw the cache.  That must be done separately.
542	
543	
544	 (*) Invoke the retrieval I/O completion function:
545	
546		void fscache_end_io(struct fscache_retrieval *op, struct page *page,
547				    int error);
548	
549	     This is called to note the end of an attempt to retrieve a page.  The
550	     error value should be 0 if successful and an error otherwise.
551	
552	
553	 (*) Record that one or more pages being retrieved or allocated have been dealt
554	     with:
555	
556		void fscache_retrieval_complete(struct fscache_retrieval *op,
557						int n_pages);
558	
559	     This is called to record the fact that one or more pages have been dealt
560	     with and are no longer the concern of this operation.  When the number of
561	     pages remaining in the operation reaches 0, the operation will be
562	     completed.
563	
564	
565	 (*) Record operation completion:
566	
567		void fscache_op_complete(struct fscache_operation *op);
568	
569	     This is called to record the completion of an operation.  This deducts
570	     this operation from the parent object's run state, potentially permitting
571	     one or more pending operations to start running.
572	
573	
574	 (*) Set highest store limit:
575	
576		void fscache_set_store_limit(struct fscache_object *object,
577					     loff_t i_size);
578	
579	     This sets the limit FS-Cache imposes on the highest byte it's willing to
580	     try and store for a netfs.  Any page over this limit is automatically
581	     rejected by fscache_read_alloc_page() and co with -ENOBUFS.
582	
583	
584	 (*) Mark pages as being cached:
585	
586		void fscache_mark_pages_cached(struct fscache_retrieval *op,
587					       struct pagevec *pagevec);
588	
589	     This marks a set of pages as being cached.  After this has been called,
590	     the netfs must call fscache_uncache_page() to unmark the pages.
591	
592	
593	 (*) Perform coherency check on an object:
594	
595		enum fscache_checkaux fscache_check_aux(struct fscache_object *object,
596							const void *data,
597							uint16_t datalen);
598	
599	     This asks the netfs to perform a coherency check on an object that has
600	     just been looked up.  The cookie attached to the object will determine the
601	     netfs to use.  data and datalen should specify where the auxiliary data
602	     retrieved from the cache can be found.
603	
604	     One of three values will be returned:
605	
606		(*) FSCACHE_CHECKAUX_OKAY
607	
608		    The coherency data indicates the object is valid as is.
609	
610		(*) FSCACHE_CHECKAUX_NEEDS_UPDATE
611	
612		    The coherency data needs updating, but otherwise the object is
613		    valid.
614	
615		(*) FSCACHE_CHECKAUX_OBSOLETE
616	
617		    The coherency data indicates that the object is obsolete and should
618		    be discarded.
619	
620	
621	 (*) Initialise a freshly allocated object:
622	
623		void fscache_object_init(struct fscache_object *object);
624	
625	     This initialises all the fields in an object representation.
626	
627	
628	 (*) Indicate the destruction of an object:
629	
630		void fscache_object_destroyed(struct fscache_cache *cache);
631	
632	     This must be called to inform FS-Cache that an object that belonged to a
633	     cache has been destroyed and deallocated.  This will allow continuation
634	     of the cache withdrawal process when it is stopped pending destruction of
635	     all the objects.
636	
637	
638	 (*) Indicate negative lookup on an object:
639	
640		void fscache_object_lookup_negative(struct fscache_object *object);
641	
642	     This is called to indicate to FS-Cache that a lookup process for an object
643	     found a negative result.
644	
645	     This changes the state of an object to permit reads pending on lookup
646	     completion to go off and start fetching data from the netfs server as it's
647	     known at this point that there can't be any data in the cache.
648	
649	     This may be called multiple times on an object.  Only the first call is
650	     significant - all subsequent calls are ignored.
651	
652	
653	 (*) Indicate an object has been obtained:
654	
655		void fscache_obtained_object(struct fscache_object *object);
656	
657	     This is called to indicate to FS-Cache that a lookup process for an object
658	     produced a positive result, or that an object was created.  This should
659	     only be called once for any particular object.
660	
661	     This changes the state of an object to indicate:
662	
663		(1) if no call to fscache_object_lookup_negative() has been made on
664		    this object, that there may be data available, and that reads can
665		    now go and look for it; and
666	
667	        (2) that writes may now proceed against this object.
668	
669	
670	 (*) Indicate that object lookup failed:
671	
672		void fscache_object_lookup_error(struct fscache_object *object);
673	
674	     This marks an object as having encountered a fatal error (usually EIO)
675	     and causes it to move into a state whereby it will be withdrawn as soon
676	     as possible.
677	
678	
679	 (*) Get and release references on a retrieval record:
680	
681		void fscache_get_retrieval(struct fscache_retrieval *op);
682		void fscache_put_retrieval(struct fscache_retrieval *op);
683	
684	     These two functions are used to retain a retrieval record whilst doing
685	     asynchronous data retrieval and block allocation.
686	
687	
688	 (*) Enqueue a retrieval record for processing.
689	
690		void fscache_enqueue_retrieval(struct fscache_retrieval *op);
691	
692	     This enqueues a retrieval record for processing by the FS-Cache thread
693	     pool.  One of the threads in the pool will invoke the retrieval record's
694	     op->op.processor callback function.  This function may be called from
695	     within the callback function.
696	
697	
698	 (*) List of object state names:
699	
700		const char *fscache_object_states[];
701	
702	     For debugging purposes, this may be used to turn the state that an object
703	     is in into a text string for display purposes.
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.