Interface LLMLingua2Config

Configuration for LLMLingua-2 internals.

interface LLMLingua2Config {
    maxBatchSize: number;
    maxForceToken: number;
    maxSeqLen: number;
}

Properties

maxBatchSize: number

Maximum batch size for processing prompts.

maxForceToken: number

Maximum number of tokens to force in the compression.

maxSeqLen: number

Maximum sequence length for the model.