About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / networking / strparser.txt




Custom Search

Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.

1	Stream Parser (strparser)
2	
3	Introduction
4	============
5	
6	The stream parser (strparser) is a utility that parses messages of an
7	application layer protocol running over a data stream. The stream
8	parser works in conjunction with an upper layer in the kernel to provide
9	kernel support for application layer messages. For instance, Kernel
10	Connection Multiplexor (KCM) uses the Stream Parser to parse messages
11	using a BPF program.
12	
13	The strparser works in one of two modes: receive callback or general
14	mode.
15	
16	In receive callback mode, the strparser is called from the data_ready
17	callback of a TCP socket. Messages are parsed and delivered as they are
18	received on the socket.
19	
20	In general mode, a sequence of skbs are fed to strparser from an
21	outside source. Message are parsed and delivered as the sequence is
22	processed. This modes allows strparser to be applied to arbitrary
23	streams of data.
24	
25	Interface
26	=========
27	
28	The API includes a context structure, a set of callbacks, utility
29	functions, and a data_ready function for receive callback mode. The
30	callbacks include a parse_msg function that is called to perform
31	parsing (e.g.  BPF parsing in case of KCM), and a rcv_msg function
32	that is called when a full message has been completed.
33	
34	Functions
35	=========
36	
37	strp_init(struct strparser *strp, struct sock *sk,
38		  const struct strp_callbacks *cb)
39	
40	     Called to initialize a stream parser. strp is a struct of type
41	     strparser that is allocated by the upper layer. sk is the TCP
42	     socket associated with the stream parser for use with receive
43	     callback mode; in general mode this is set to NULL. Callbacks
44	     are called by the stream parser (the callbacks are listed below).
45	
46	void strp_pause(struct strparser *strp)
47	
48	     Temporarily pause a stream parser. Message parsing is suspended
49	     and no new messages are delivered to the upper layer.
50	
51	void strp_pause(struct strparser *strp)
52	
53	     Unpause a paused stream parser.
54	
55	void strp_stop(struct strparser *strp);
56	
57	     strp_stop is called to completely stop stream parser operations.
58	     This is called internally when the stream parser encounters an
59	     error, and it is called from the upper layer to stop parsing
60	     operations.
61	
62	void strp_done(struct strparser *strp);
63	
64	     strp_done is called to release any resources held by the stream
65	     parser instance. This must be called after the stream processor
66	     has been stopped.
67	
68	int strp_process(struct strparser *strp, struct sk_buff *orig_skb,
69			 unsigned int orig_offset, size_t orig_len,
70			 size_t max_msg_size, long timeo)
71	
72	    strp_process is called in general mode for a stream parser to
73	    parse an sk_buff. The number of bytes processed or a negative
74	    error number is returned. Note that strp_process does not
75	    consume the sk_buff. max_msg_size is maximum size the stream
76	    parser will parse. timeo is timeout for completing a message.
77	
78	void strp_data_ready(struct strparser *strp);
79	
80	    The upper layer calls strp_tcp_data_ready when data is ready on
81	    the lower socket for strparser to process. This should be called
82	    from a data_ready callback that is set on the socket. Note that
83	    maximum messages size is the limit of the receive socket
84	    buffer and message timeout is the receive timeout for the socket.
85	
86	void strp_check_rcv(struct strparser *strp);
87	
88	    strp_check_rcv is called to check for new messages on the socket.
89	    This is normally called at initialization of a stream parser
90	    instance or after strp_unpause.
91	
92	Callbacks
93	=========
94	
95	There are six callbacks:
96	
97	int (*parse_msg)(struct strparser *strp, struct sk_buff *skb);
98	
99	    parse_msg is called to determine the length of the next message
100	    in the stream. The upper layer must implement this function. It
101	    should parse the sk_buff as containing the headers for the
102	    next application layer message in the stream.
103	
104	    The skb->cb in the input skb is a struct strp_msg. Only
105	    the offset field is relevant in parse_msg and gives the offset
106	    where the message starts in the skb.
107	
108	    The return values of this function are:
109	
110	    >0 : indicates length of successfully parsed message
111	    0  : indicates more data must be received to parse the message
112	    -ESTRPIPE : current message should not be processed by the
113	          kernel, return control of the socket to userspace which
114	          can proceed to read the messages itself
115	    other < 0 : Error in parsing, give control back to userspace
116	          assuming that synchronization is lost and the stream
117	          is unrecoverable (application expected to close TCP socket)
118	
119	    In the case that an error is returned (return value is less than
120	    zero) and the parser is in receive callback mode, then it will set
121	    the error on TCP socket and wake it up. If parse_msg returned
122	    -ESTRPIPE and the stream parser had previously read some bytes for
123	    the current message, then the error set on the attached socket is
124	    ENODATA since the stream is unrecoverable in that case.
125	
126	void (*lock)(struct strparser *strp)
127	
128	    The lock callback is called to lock the strp structure when
129	    the strparser is performing an asynchronous operation (such as
130	    processing a timeout). In receive callback mode the default
131	    function is to lock_sock for the associated socket. In general
132	    mode the callback must be set appropriately.
133	
134	void (*unlock)(struct strparser *strp)
135	
136	    The unlock callback is called to release the lock obtained
137	    by the lock callback. In receive callback mode the default
138	    function is release_sock for the associated socket. In general
139	    mode the callback must be set appropriately.
140	
141	void (*rcv_msg)(struct strparser *strp, struct sk_buff *skb);
142	
143	    rcv_msg is called when a full message has been received and
144	    is queued. The callee must consume the sk_buff; it can
145	    call strp_pause to prevent any further messages from being
146	    received in rcv_msg (see strp_pause above). This callback
147	    must be set.
148	
149	    The skb->cb in the input skb is a struct strp_msg. This
150	    struct contains two fields: offset and full_len. Offset is
151	    where the message starts in the skb, and full_len is the
152	    the length of the message. skb->len - offset may be greater
153	    then full_len since strparser does not trim the skb.
154	
155	int (*read_sock_done)(struct strparser *strp, int err);
156	
157	     read_sock_done is called when the stream parser is done reading
158	     the TCP socket in receive callback mode. The stream parser may
159	     read multiple messages in a loop and this function allows cleanup
160	     to occur when exiting the loop. If the callback is not set (NULL
161	     in strp_init) a default function is used.
162	
163	void (*abort_parser)(struct strparser *strp, int err);
164	
165	     This function is called when stream parser encounters an error
166	     in parsing. The default function stops the stream parser and
167	     sets the error in the socket if the parser is in receive callback
168	     mode. The default function can be changed by setting the callback
169	     to non-NULL in strp_init.
170	
171	Statistics
172	==========
173	
174	Various counters are kept for each stream parser instance. These are in
175	the strp_stats structure. strp_aggr_stats is a convenience structure for
176	accumulating statistics for multiple stream parser instances.
177	save_strp_stats and aggregate_strp_stats are helper functions to save
178	and aggregate statistics.
179	
180	Message assembly limits
181	=======================
182	
183	The stream parser provide mechanisms to limit the resources consumed by
184	message assembly.
185	
186	A timer is set when assembly starts for a new message. In receive
187	callback mode the message timeout is taken from rcvtime for the
188	associated TCP socket. In general mode, the timeout is passed as an
189	argument in strp_process. If the timer fires before assembly completes
190	the stream parser is aborted and the ETIMEDOUT error is set on the TCP
191	socket if in receive callback mode.
192	
193	In receive callback mode, message length is limited to the receive
194	buffer size of the associated TCP socket. If the length returned by
195	parse_msg is greater than the socket buffer size then the stream parser
196	is aborted with EMSGSIZE error set on the TCP socket. Note that this
197	makes the maximum size of receive skbuffs for a socket with a stream
198	parser to be 2*sk_rcvbuf of the TCP socket.
199	
200	In general mode the message length limit is passed in as an argument
201	to strp_process.
202	
203	Author
204	======
205	
206	Tom Herbert (tom@quantonium.net)
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.