Body Factory Refactoring¶
The core logic used to generate proxy replies is convoluted and fragile. It has needed to be cleaned up for a long time both to make it more robust and maintainable but also to add features that are currently hard to do. There are also arbitrary internal limits which are entirely artifacts of the implementation.
The goal is to replace all of this with
MIOBuffer to get the following benefits:
Remove arbitrary limits.
Remove all direct memory allocation, use only
Reduce costs of using the generated bodies later. An
MIOBuffercan be directly fed to an output tunnel or even a socket without additional copies or allocations.
Simplify the code. All output is done directly to an
The basic plan is to extend the
BufferWriter class to have one that takes an
MIOBuffer as its output target. This would make it basically equivalent to C++ I/O stream operators
but using Traffic Server core mechanisms.
The starting change is
ats_scoped_str. This is used to populate the
Content-Typeheader and so doesn’t need to be a variable sized buffer. It could probably be made a fixed size string of some reasonable length (~128 bytes) without loss of generality.
This has a large and extensive ripple of changes that achieve the goals of this project. Chasing these down and fixing them will be the bulk of the effort. There are a few others issues to be careful about
Some mark is needed to track if there is a body at all - there is a semantic difference between a body of length zero and no body at all.
A new “Format” class could be done which processes the current log style format strings in a generic and modular way. The body factory would be updated to use these new classes.
The following sections are intended as documentation to be added to the Traffic Server documentation set.
Apply the log format string to the transaction txnp placing the result in the buffer out. The format is identical to custom log formats. The result is appended to out in ASCII format. out must be created by the plugin.
Sets an error type body to a transaction.
Note that both string arguments must be allocated with
TSstrdup(). The mimetype is optional, and if not provided it
text/html. Sending an empty string would prevent setting
a content type header (but that is not adviced).
Traffic Server IO buffer reader API.
int (*TSIOBufferBlockFunc)(void const* data, int64_t nbytes, void* context)
data is the data in the
TSIOBufferBlockand is nbytes long. context is opaque data provided to the API call along with this function and passed on to the function. This function should return
trueto continue iteration or
falseto terminate iteration.
TSIOBufferReader is an read accessor for
TSIOBuffer. It represents a location in
the contents of the buffer. A buffer can have multiple readers and each reader consumes data in the
buffer independently. Data which for which there are no readers is discarded from the buffer. This
has two very important consequences –
- Data for which there are no readers and no writer will be discarded. In effect this means without
any readers only the current write buffer block will be maintained, older buffer blocks will disappear.
Conversely keeping a reader around unused will pin the buffer data in memory. This can be useful or harmful.
A buffer has a fixed amount of possible readers (currently 5) which is determined at compile time. Reader allocation is fast and cheap until this maxium is reached at which point it fails.
TSIOBufferReaderAlloc()allocates a reader for the IO buffer bufp. This should only be
called on a newly allocated buffer. If not the location of the reader in the buffer will be indeterminate. Use
TSIOBufferReaderClone()to get another reader if the buffer is already in use.
TSIOBufferReaderClone() allocates a reader and sets its position in the buffer to be the same as reader.
TSIOBufferReaderFree()de-allocates the reader. Any data referenced only by this reader is
discarded from the buffer.
TSIOBufferReaderConsume()advances the position of reader in its IO buffer by the
the smaller of nbytes and the maximum available in the buffer.
TSIOBufferReaderStart() returns the IO buffer block containing the position of
The byte at the position of reader is in the block but is not necessarily the first byte of the block.
TSIOBufferReaderAvail()returns the number of bytes which reader could consume. That is
the number of bytes in the IO buffer starting at the current position of reader.
trueif the available number of bytes for
reader is at least nbytes,
falseif not. This can be more efficient than
TSIOBufferReaderAvail()because the latter must walk all the IO buffer blocks in the IO buffer. This function returns as soon as the return value can be determined. In particular a value of
1for nbytes means only the first buffer block will be checked.
TSIOBufferReaderRead()reads data from reader. This first copies data from the IO
buffer for reader to the target buffer bufp, starting at reader`s position, and then advances (as with :func:`TSIOBufferReaderConsume) reader`s position past the copied data. The amount of data read in this fashion is the smaller of the amount of data available in the IO buffer for :arg:`reader and the size of the target buffer (length).
TSIOBufferReaderIterate() iterates over the blocks for reader. For each block
func is called with with the data for the block and context. The context is an
opaque type to this function and is passed unchanged to func. It is intended to be used as
context for func. If func returns
false the iteration terminates. If func
returns true the block is consumed. The return value for
TSIOBufferReaderIterate() is the
return value from the last call to func.
If it would be a problem for the iteration to consume the data (especially in cases where
falsemight be returned) the reader can be cloned via
TSIOBufferReaderClone()to keep the data in the IO buffer and available. If not needed the reader can be destroyed or if needed the original reader can be destroyed and replaced by the clone.
TSIOBuffer will de-allocate and destroy all readers for that buffer.