State-of-the-Art Retrieval Augmented Generation at Scale in Spark NLP

Current RAG LLM systems struggle to efficiently scale the number or size of processed documents or handle the complex pre- and post-processing pipelines needed when going from POC to production. This session shows how the open source Spark NLP library addresses these issues:

• Natively scale pre-processing pipelines to handle multimodal inputs, document segmentation, semantic sentence, paragraph splitting, and data normalization challenges.
• Calculate state-of-the-art text embeddings, which are then loaded into a vector database, several times faster than Hugging Face on a single machine or an order of magnitude faster & cheaper on a commodity cluster.
• Provide post-processing modules such as reranking, post-filtering, expansion, augmentation, or keyword extraction without requiring other libraries.
• Use the native integration with LangChain and HayStack when these libraries are being used.

This is a session for data scientists building production-grade LLM systems.