Class PromptCompressor

The TypeScript implementation on original PromptCompressor, which is a class for compressing prompts using a language model.

See

Original Implementation

Index

Constructors

Methods

compress compress_prompt

Constructors

constructor

new PromptCompressor(
    model: PreTrainedModel,
    tokenizer: PreTrainedTokenizer,
    getPureToken: GetPureTokenFunction,
    isBeginOfNewWord: IsBeginOfNewWordFunction,
    oaiTokenizer: Tiktoken,
    llmlingua2Config?: {
        max_batch_size: number;
        max_force_token: number;
        max_seq_length: number;
    },
    logger?: Logger,
): PromptCompressor
Parameters
- model: PreTrainedModel
  The pre-trained model to use for compression.
- tokenizer: PreTrainedTokenizer
  The pre-trained tokenizer to use for compression.
- getPureToken: GetPureTokenFunction
  Function to get the pure token from a token. This is used to normalize tokens before processing.
- isBeginOfNewWord: IsBeginOfNewWordFunction
  Function to check if a token is the beginning of a new word. This is used to determine how to merge tokens into words.
- oaiTokenizer: Tiktoken
  The tokenizer to use calculating the compression rate.
- llmlingua2Config: { max_batch_size: number; max_force_token: number; max_seq_length: number } = ...
  Configuration for LLMLingua2.
  - max_batch_size: number
    Maximum batch size for processing prompts. This is used to limit the number of prompts processed in a single batch.
  - max_force_token: number
    Maximum number of tokens to force in the compression. This is used to ensure that certain tokens are always included in the compressed prompt.
  - max_seq_length: number
    Maximum sequence length for the model. This is used to limit the length of the input sequences to the model.
- logger: Logger = console.log
  Logger function to log messages.
Returns PromptCompressor
- Defined in prompt-compressor.ts:171

Methods

compress

compress(
context: string,
__namedParameters: CompressPromptOptions,
): Promise<string>
Compresses a prompt based on the given options.
Parameters
- context: string
- __namedParameters: CompressPromptOptions
Returns Promise<string>
- Defined in prompt-compressor.ts:245

compress_prompt

compress_prompt(
context: string,
options: CompressPromptOptionsSnakeCase,
): Promise<string>
Compresses a prompt based on the given options. Alias for compress, but uses snake_case for options.
Parameters
- context: string
- options: CompressPromptOptionsSnakeCase
Returns Promise<string>
Alias
compress
- Defined in prompt-compressor.ts:274