org.jlab.coda.hipo

Class Writer

java.lang.Object

org.jlab.coda.hipo.Writer

All Implemented Interfaces:

java.lang.AutoCloseable

public class Writer
extends java.lang.Object
implements java.lang.AutoCloseable

Class to write Evio/HIPO files. This class replaces the Writer class with a more efficient writing system. Two asynchronous writes are done simultaneously in this class for greater file writing speed.

Since:

6.0 8/10/17

Version:

6.0

Author:

gavalian, timmer

Constructor Summary

Constructors

Constructor and Description
Writer() Default constructor.
Writer(java.nio.ByteBuffer buf) Constructor for writing to a ByteBuffer.
Writer(java.nio.ByteBuffer buf, byte[] userHeader) Constructor for writing to a ByteBuffer with a user header.
Writer(java.nio.ByteBuffer buf, int maxEventCount, int maxBufferSize, java.lang.String dictionary, byte[] firstEvent) Constructor with byte order.
Writer(java.nio.ByteOrder order, int maxEventCount, int maxBufferSize) Constructor with byte order.
Writer(HeaderType hType, java.nio.ByteOrder order, int maxEventCount, int maxBufferSize) Constructor with byte order and header type.
Writer(HeaderType hType, java.nio.ByteOrder order, int maxEventCount, int maxBufferSize, java.lang.String dictionary, byte[] firstEvent, CompressionType compType, boolean addTrailerIndex) Constructor with byte order.
Writer(java.lang.String filename) Constructor with filename.
Writer(java.lang.String filename, java.nio.ByteOrder order, int maxEventCount, int maxBufferSize) Constructor with filename and byte order.

Method Summary

Modifier and Type	Method and Description
void	addEvent(byte[] buffer) Add a byte array to the internal record.
void	addEvent(byte[] buffer, int offset, int length) Add a byte array to the internal record.
void	addEvent(java.nio.ByteBuffer buffer) Add a ByteBuffer to the internal record.
void	addEvent(EvioBank bank) Add an EvioBank to the internal record.
void	addEvent(EvioNode node) Add an EvioNode to the internal record.
boolean	addTrailer() Does this writer add a trailer to the end of the file/buffer?
void	addTrailer(boolean add) Set whether this writer adds a trailer to the end of the file/buffer.
boolean	addTrailerWithIndex() Does this writer add a trailer with a record index to the end of the file? Or, if writing to a buffer, is a trailer added with no index?
void	addTrailerWithIndex(boolean addTrailingIndex) Set whether this writer adds a trailer with a record index to the end of the file.
void	close() Close opened file.
java.nio.ByteBuffer	createHeader(byte[] userHeader) Create and return a buffer containing a general file header followed by the user header given in the argument.
java.nio.ByteBuffer	createHeader(java.nio.ByteBuffer userHeader) Create and return a buffer containing a general file header followed by the user header given in the argument.
static java.nio.ByteBuffer	createRecord(java.lang.String dictionary, byte[] firstEvent, java.nio.ByteOrder byteOrder, FileHeader fileHeader, RecordHeader recordHeader) Create a buffer representation of a record containing dictionary and/or first event.
java.nio.ByteBuffer	getBuffer() Get the buffer being written to.
java.nio.ByteOrder	getByteOrder() Get the file's byte order.
CompressionType	getCompressionType() Convenience method that gets compression type for the file being written.
FileHeader	getFileHeader() Get the file header.
void	open(java.nio.ByteBuffer buf, byte[] userHeader) Specify a buffer and write first record header with given user header.
void	open(java.nio.ByteBuffer buf, byte[] userHdr, int off, int len) Specify a buffer and write first record header with given user header.
void	open(java.lang.String filename) Open a new file and write file header with no user header.
void	open(java.lang.String filename, byte[] userHeader) Open a file and write file header with given user header.
void	reset() Get this object ready for re-use.
Writer	setCompressionType(CompressionType compression) Convenience method that sets compression type for the file.
void	writeRecord(RecordOutputStream record) Appends the record to the file.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Writer

public Writer()

Default constructor. No file is opened. Any file will have little endian byte order.

Writer

public Writer(java.nio.ByteOrder order,
              int maxEventCount,
              int maxBufferSize)

Constructor with byte order. No file is opened. File header type is evio file (HeaderType.EVIO_FILE).

Parameters:

order - byte order of written file. Little endian if null.

maxEventCount - max number of events a record can hold. Value of O means use default (1M).

maxBufferSize - max number of uncompressed data bytes a record can hold. Value of < 8MB results in default of 8MB.

Writer

public Writer(HeaderType hType,
              java.nio.ByteOrder order,
              int maxEventCount,
              int maxBufferSize)

Constructor with byte order and header type. No file is opened.

Parameters:

hType - the type of the file. If set to HeaderType.HIPO_FILE, the header will be written with the first 4 bytes set to HIPO.

order - byte order of written file. Little endian if null.

maxEventCount - max number of events a record can hold. Value of O means use default (1M).

maxBufferSize - max number of uncompressed data bytes a record can hold. Value of < 8MB results in default of 8MB.

Writer

public Writer(HeaderType hType,
              java.nio.ByteOrder order,
              int maxEventCount,
              int maxBufferSize,
              java.lang.String dictionary,
              byte[] firstEvent,
              CompressionType compType,
              boolean addTrailerIndex)

Constructor with byte order. The given file is opened so any subsequent call to open will fail. This method places the dictionary and first event into the file header's user header.

Parameters:

hType - the type of the file. If set to HeaderType.HIPO_FILE, the header will be written with the first 4 bytes set to HIPO.

order - byte order of written file. Little endian if null.

maxEventCount - max number of events a record can hold. Value of O means use default (1M).

maxBufferSize - max number of uncompressed data bytes a record can hold. Value of < 8MB results in default of 8MB.

dictionary - string holding an evio format dictionary to be placed in userHeader.

firstEvent - byte array containing an evio event to be included in userHeader. It must be in the same byte order as the order argument.

compType - type of data compression to use.

addTrailerIndex - if true, we add a record index to the trailer.

Writer

public Writer(java.lang.String filename)
       throws java.io.IOException

Constructor with filename. The output file will be created with no user header and therefore open(String) is called in this method and should not be called again. File byte order is little endian.

Parameters:

filename - output file name

Throws:

java.io.IOException - if file cannot be found or IO error writing to file

Writer

public Writer(java.lang.String filename,
              java.nio.ByteOrder order,
              int maxEventCount,
              int maxBufferSize)
       throws java.io.IOException

Constructor with filename and byte order. The output file will be created with no user header and therefore open(String) is called in this method and should not be called again. LZ4 compression.

Parameters:

filename - output file name

order - byte order of written file or null for default (little endian)

maxEventCount - max number of events a record can hold. Value of O means use default (1M).

maxBufferSize - max number of uncompressed data bytes a record can hold. Value of < 8MB results in default of 8MB.

Throws:

java.io.IOException - if file cannot be found or IO error writing to file

Writer

public Writer(java.nio.ByteBuffer buf)
       throws HipoException

Constructor for writing to a ByteBuffer. Byte order is taken from the buffer. No compression.

Parameters:

buf - buffer in to which to write events and/or records.

Throws:

HipoException - if buf arg is null.

Writer

public Writer(java.nio.ByteBuffer buf,
              byte[] userHeader)
       throws HipoException

Constructor for writing to a ByteBuffer with a user header. Byte order is taken from the buffer. No compression.

Parameters:

buf - buffer in to which to write events and/or records.

userHeader - array containing user header.

Throws:

HipoException - if buf arg is null.

Writer

public Writer(java.nio.ByteBuffer buf,
              int maxEventCount,
              int maxBufferSize,
              java.lang.String dictionary,
              byte[] firstEvent)
       throws HipoException

Constructor with byte order. This method places the dictionary and first event into the file header's user header. No compression.

Parameters:

buf - buffer in to which to write events and/or records.

maxEventCount - max number of events a record can hold. Value of O means use default (1M).

maxBufferSize - max number of uncompressed data bytes a record can hold. Value of < 8MB results in default of 8MB.

dictionary - string holding an evio format dictionary to be placed in userHeader.

firstEvent - byte array containing an evio event to be included in userHeader. It must be in the same byte order as the order argument.

Throws:

HipoException - if buf arg is null.

Method Detail

getBuffer

public java.nio.ByteBuffer getBuffer()

Get the buffer being written to.

Returns:

buffer being written to.

getByteOrder

public java.nio.ByteOrder getByteOrder()

Get the file's byte order.

Returns:

file's byte order.

getFileHeader

public FileHeader getFileHeader()

Get the file header.

Returns:

file header.

getCompressionType

public CompressionType getCompressionType()

Convenience method that gets compression type for the file being written.

Returns:

compression type for the file being written.

setCompressionType

public final Writer setCompressionType(CompressionType compression)

Convenience method that sets compression type for the file. The compression type is also set for internal record(s). When writing to the file, record data will be compressed according to the given type.

Parameters:

compression - compression type

Returns:

this object

addTrailer

public boolean addTrailer()

Does this writer add a trailer to the end of the file/buffer?

Returns:

true if this writer adds a trailer to the end of the file/buffer, else false.

addTrailer

public void addTrailer(boolean add)

Set whether this writer adds a trailer to the end of the file/buffer.

Parameters:

add - if true, at the end of file/buffer, add an ending header (trailer) with no index of records and no following data. Update the file header to contain a file offset to the trailer.

addTrailerWithIndex

public boolean addTrailerWithIndex()

Does this writer add a trailer with a record index to the end of the file? Or, if writing to a buffer, is a trailer added with no index?

Returns:

if writing to a file: true if this writer adds a trailer with a record index to the end of the file, else false. If writing to a buffer, true if this writer adds a traile to the end of the buffer, else false.

addTrailerWithIndex

public void addTrailerWithIndex(boolean addTrailingIndex)

Set whether this writer adds a trailer with a record index to the end of the file. Or, if writing to a buffer, set whether a trailer is added with no index.

Parameters:

addTrailingIndex - if true, at the end of file, add an ending header (trailer) with an index of all records but with no following data. Update the file header to contain a file offset to the trailer. If true, and writing to a buffer, add a trailer with no index to the end of the buffer.

open

public final void open(java.lang.String filename)
                throws HipoException,
                       java.io.IOException

Open a new file and write file header with no user header.

Parameters:

filename - output file name

Throws:

HipoException - if open already called without being followed by calling close.

java.io.IOException - if file cannot be found or IO error writing to file

open

public final void open(java.lang.String filename,
                       byte[] userHeader)
                throws HipoException,
                       java.io.IOException

Open a file and write file header with given user header. User header is automatically padded when written.

Parameters:

filename - name of file to write to.

userHeader - byte array representing the optional user's header. If this is null AND dictionary and/or first event are given, the dictionary and/or first event will be placed in its own record and written as the user header.

Throws:

HipoException - filename arg is null, if constructor specified writing to a buffer, or if open() was already called without being followed by reset().

java.io.IOException - if file cannot be found or IO error writing to file

open

public final void open(java.nio.ByteBuffer buf,
                       byte[] userHeader)
                throws HipoException

Specify a buffer and write first record header with given user header. User header is automatically padded when written.

Parameters:

buf - buffer to writer to.

userHeader - byte array representing the optional user's header. Warning: this will not be used until first record is written! So don't go changing it in the meantime! If this is null AND dictionary and/or first event are given, the dictionary and/or first event will be placed in its own record and written as the user header of the first record's header.

Throws:

HipoException - buf arg is null, if constructor specified writing to a file, or if open() was already called without being followed by reset().

open

public final void open(java.nio.ByteBuffer buf,
                       byte[] userHdr,
                       int off,
                       int len)
                throws HipoException

Specify a buffer and write first record header with given user header. User header is automatically padded when written.

Parameters:

buf - buffer to writer to.

userHdr - byte array representing the optional user's header. Warning: this will not be used until first record is written! So don't go changing it in the meantime! If this is null AND dictionary and/or first event are given, the dictionary and/or first event will be placed in its own record and written as the user header of the first record's header.

off - offset of valid data in userHdr.

len - length of valid data (bytes) in userHdr (starting at off).

Throws:

HipoException - buf arg is null, if constructor specified writing to a file, or if open() was already called without being followed by reset().

createRecord

public static java.nio.ByteBuffer createRecord(java.lang.String dictionary,
                                               byte[] firstEvent,
                                               java.nio.ByteOrder byteOrder,
                                               FileHeader fileHeader,
                                               RecordHeader recordHeader)

Create a buffer representation of a record containing dictionary and/or first event. No compression.

Parameters:

dictionary - dictionary xml string

firstEvent - bytes representing evio event

byteOrder - byte order of returned byte array

fileHeader - file header to update with dictionary/first-event info (may be null).

recordHeader - record header to update with dictionary info (may be null).

Returns:

buffer representation of record containing dictionary and/or first event. Null if both are null.

createHeader

public java.nio.ByteBuffer createHeader(byte[] userHeader)
                                 throws HipoException

Create and return a buffer containing a general file header followed by the user header given in the argument. If user header is not padded to 4-byte boundary, it's done here.

Parameters:

userHeader - byte array containing a user-defined header, may be null.

Returns:

buffer containing a file header followed by the user-defined header.

Throws:

HipoException - if writing to buffer.

createHeader

public java.nio.ByteBuffer createHeader(java.nio.ByteBuffer userHeader)
                                 throws HipoException

Create and return a buffer containing a general file header followed by the user header given in the argument. If user header is not padded to 4-byte boundary, it's done here.

Parameters:

userHeader - buffer containing a user-defined header which must be READY-TO-READ!

Returns:

buffer containing a file header followed by the user-defined header.

Throws:

HipoException - if writing to buffer.

writeRecord

public void writeRecord(RecordOutputStream record)
                 throws java.io.IOException

Appends the record to the file. Using this method in conjunction with addEvent() is not thread-safe.

Parameters:

record - record object

Throws:

java.lang.IllegalArgumentException - if arg's byte order is opposite to output endian.

java.io.IOException - if error writing to file.

addEvent

public void addEvent(byte[] buffer)
              throws java.io.IOException

Add a byte array to the internal record. If the length of the buffer exceeds the maximum size of the record, the record will be written to the file (compressed if the flag is set). Internal record will be reset to receive new buffers. Using this method in conjunction with writeRecord() is not thread-safe. The byte order of event's byte array must match the byte order given in constructor!

Parameters:

buffer - array to add to the file.

Throws:

java.io.IOException - if cannot write to file.

addEvent

public void addEvent(byte[] buffer,
                     int offset,
                     int length)
              throws java.io.IOException

Add a byte array to the internal record. If the length of the buffer exceeds the maximum size of the record, the record will be written to the file (compressed if the flag is set). Internal record will be reset to receive new buffers. Using this method in conjunction with writeRecord() is not thread-safe. The byte order of event's byte array must match the byte order given in constructor!

Parameters:

buffer - array to add to the file.

offset - offset into array from which to start writing data.

length - number of bytes to write from array.

Throws:

java.io.IOException - if cannot write to file.

addEvent

public void addEvent(java.nio.ByteBuffer buffer)
              throws java.io.IOException,
                     HipoException

Add a ByteBuffer to the internal record. If the length of the buffer exceeds the maximum size of the record, the record will be written to the file (compressed if the flag is set). Internal record will be reset to receive new buffers. Using this method in conjunction with writeRecord() is not thread-safe. The byte order of event's data must match the byte order given in constructor!

Parameters:

buffer - array to add to the file.

Throws:

HipoException - if buf arg's byte order is wrong.

java.io.IOException - if cannot write to file.

addEvent

public void addEvent(EvioBank bank)
              throws java.io.IOException

Add an EvioBank to the internal record. If the length of the bank exceeds the maximum size of the record, the record will be written to the file (compressed if the flag is set). Internal record will be reset to receive new buffers. Using this method in conjunction with writeRecord() is not thread-safe.

Parameters:

bank - event to add to the file.

Throws:

java.io.IOException - if cannot write to file.

addEvent

public void addEvent(EvioNode node)
              throws HipoException,
                     java.io.IOException

Add an EvioNode to the internal record. If the length of the data exceeds the maximum size of the record, the record will be written to the file (compressed if the flag is set). Internal record will be reset to receive new buffers. Using this method in conjunction with writeRecord() is not thread-safe. The byte order of node's data must match the byte order given in constructor!

Parameters:

node - node to add to the file.

Throws:

HipoException - if node does not correspond to a bank.

java.io.IOException - if cannot write to file.

reset

public void reset()

Get this object ready for re-use. Follow calling this with call to open(String).

close

public void close()
           throws java.io.IOException

Close opened file. If the output record contains events, they will be flushed to file/buffer. Trailer and its optional index written if requested.

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.io.IOException - if error writing to file