TruLens provides a set of tools for developing and monitoring neural nets, including large language models. This includes both tools for evaluation of LLMs and LLM-based applications with TruLens-Eval and deep learning explainability with TruLens-Explain. TruLens-Eval and TruLens-Explain are housed in separate packages and can be used independently.
| General | |
|---|---|
| Author | TruEra |
| Repository | https://github.com/truera/trulens |
| Type | LLMs Development and Explainability Tool |
TruLens-Eval
TruLens-Eval contains instrumentation and evaluation tools for large language model (LLM) based applications. It supports the iterative development and monitoring of a wide range of LLM applications by wrapping your application to log key metadata across the entire chain (or off chain if your project does not use chains) on your local machine. Importantly, it also gives you the tools you need to evaluate the quality of your LLM-based applications.
Key Features:
-
Evaluation: TruLens supports the evaluation of inputs, outputs, and internals of your LLM application using any model, including LLMs. It offers various out-of-the-box feedback functions for evaluation, such as groundedness, relevance, and toxicity. The framework is easily extensible for custom evaluation requirements.
-
Tracking: TruLens contains instrumentation for any LLM application, including question answering, retrieval-augmented generation, agent-based applications, and more. This instrumentation allows for the tracking of a wide variety of usage metrics and metadata. TruLens' instrumentation can be applied to any LLM application without being tied down to a specific framework. Additionally, deep integrations with LangChain and Llama-Index allow the capture of internal metadata and text. Anything that is tracked by the instrumentation can be evaluated!
TruLens-Explain
TruLens-Explain is a cross-framework library for deep learning explainability. It provides a uniform abstraction layer over TensorFlow, PyTorch, and Keras, allowing for input and internal explanations.
Read more about TruLens-Explain
Getting Started
TruLens User Case Guides
TruLens Tutorials
- TruLens + Google Cloud Vertex AI Tutorial: Building RAG Applications
- Evaluate and Improve your Chatbots with TruLens
- Visionary Data: Leveraging Trulens with MongoDB & LlamaIndex
- Unlocking Creativity: Integrating DALL·E 2 API with TruLens for Enhanced Image Generation
- TruLens + Google Cloud Vertex AI Tutorial: Improve the customers support
👉 Discover more TruLens Tutorials on lablab.build
.png&w=96&q=75){kind=small}
