IngestDocument

This plugin is currently in beta. While it is considered safe for use, please be aware that its API could change in ways that are not compatible with earlier versions in future releases, or it might become unsupported.

yaml
type: "io.kestra.plugin.langchain4j.rag.ingestdocument"

Examples

yaml
id: document-ingestion
namespace: company.team

tasks:
  - id: ingest
    type: io.kestra.plugin.langchain4j.rag.IngestDocument
    provider:
      type: io.kestra.plugin.langchain4j.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ secret('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.langchain4j.embeddings.KestraKVStore
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-22.md

Properties

embeddings *Elasticsearch KestraKVStore PGVector

provider *AmazonBedrock Anthropic AzureOpenAI DeepSeek GoogleGemini GoogleVertexAI MistralAI Ollama OpenAI

documentSplitter IngestDocument-DocumentSplitter

drop booleanstring

Default false

fromDocuments array

SubType

fromExternalURLs array

SubType string

fromInternalURIs array

SubType string

fromPath string

metadata object

SubType string

Outputs

embeddingStoreOutputs object

ingestedDocuments integer

inputTokenCount integer

outputTokenCount integer

totalTokenCount integer

Definitions

Google VertexAI Model Provider

endpoint *string

location *string

modelName *string

project *string

type *object

Azure OpenAI Model Provider

endpoint *string

modelName *string

type *object

apiKey string

clientId string

clientSecret string

serviceVersion string

tenantId string

Deepseek Model Provider

apiKey *string

modelName *string

type *object

baseUrl string

Default https://api.deepseek.com/v1

io.kestra.plugin.langchain4j.embeddings.Elasticsearch-ElasticsearchConnection

hosts *array

SubType string

Min items 1

basicAuth Elasticsearch-ElasticsearchConnection-BasicAuth

headers array

SubType string

pathPrefix string

strictDeprecationMode booleanstring

trustAllSsl booleanstring

Anthropic AI Model Provider

apiKey *string

modelName *string

type *object

OpenAI Model Provider

apiKey *string

modelName *string

type *object

baseUrl string

Ollama Model Provider

endpoint *string

modelName *string

type *object

io.kestra.plugin.langchain4j.embeddings.Elasticsearch-ElasticsearchConnection-BasicAuth

password string

username string

In-memory Embedding Store that then store its serialization form as a Kestra K/V pair

type *object

kvName string

Default {{flow.id}}-embedding-store

Google Gemini Model Provider

apiKey *string

modelName *string

type *object

Amazon Bedrock Model Provider

accessKeyId *string

modelName *string

secretAccessKey *string

type *object

modelType string

Default COHERE

Possible Values

COHERETITAN

io.kestra.plugin.langchain4j.rag.IngestDocument-InlineDocument

content *string

metadata object

PGVector Embedding Store

database *string

host *string

password *string

port *integerstring

table *string

type *object

user *string

useIndex booleanstring

Default false

Mistral AI Model Provider

apiKey *string

modelName *string

type *object

baseUrl string

io.kestra.plugin.langchain4j.rag.IngestDocument-DocumentSplitter

maxOverlapSizeInChars *integer

maxSegmentSizeInChars *integer

splitter string

Default RECURSIVE

Possible Values

RECURSIVEPARAGRAPHLINESENTENCEWORD

Elasticsearch Embedding Store

connection *Elasticsearch-ElasticsearchConnection

indexName *string

type *object

​Ingest​Document

IngestDocument