What is Dingo?

Dingo is a comprehensive data quality evaluation tool designed to automatically detect data quality issues in various datasets. It provides built-in rules, model evaluation methods, and supports custom evaluations, making it ideal for pre-training, fine-tuning, and evaluating datasets.

How do I install Dingo?

You can install Dingo using pip with the following command:

bash pip install dingo-python

What types of data does Dingo support?

Dingo supports commonly used text datasets and multimodal datasets, including pre-training, fine-tuning, and evaluation datasets.

Can Dingo integrate with LLMs?

Yes, Dingo supports integration with Large Language Models (LLMs) such as OpenAI, Kimi, and local models like Llama3 for advanced data quality assessments.

What are the key quality metrics that Dingo evaluates?

Dingo evaluates data quality across seven dimensions:

• Completeness
• Effectiveness
• Fluency
• Relevance
• Security
• Similarity
• Understandability

How do I run Dingo using the command line?

You can run Dingo using the command line interface (CLI) with various options. For example, to evaluate a dataset with rule sets, use:

bash python -m dingo.run.cli --input_path data.txt --dataset local -e sft --data_format plaintext --save_data True

Does Dingo offer a GUI for visualization?

Yes, Dingo generates a frontend page for visualization after evaluation (when save_data=True). You can manually start the frontend using:

bash python -m dingo.run.vsl --input output_directory

Where output_directory contains the evaluation results with a summary.json file.

Can I create custom rules with Dingo?

Yes, Dingo supports custom rules, prompts, and models. You can extend Dingo with your own rules and models to address specific data quality challenges.

What is the MCP Server in Dingo?

Dingo includes an experimental Model Context Protocol (MCP) server. MCP is an open protocol that standardizes how applications provide context to LLMs, allowing AI models to access and interact with external data sources and tools.

How does Dingo help with data governance?

Dingo assists organizations in establishing and maintaining data governance policies by providing a tool for monitoring and enforcing data quality standards.