Type Alias IsBeginOfNewWordFunction

IsBeginOfNewWordFunction: (
    token: string | null | undefined,
    force_tokens?: string[],
    token_map?: Record<string, string>,
) => boolean

Implementation on is_begin_of_new_word function of original LLMLingua implementation.

Type Declaration

- (
      token: string | null | undefined,
      force_tokens?: string[],
      token_map?: Record<string, string>,
  ): boolean
- Parameters
  - token: string | null | undefined
  - Optionalforce_tokens: string[]
  - Optionaltoken_map: Record<string, string>
  Returns boolean

See

Original Implementation