README.md

View on GitHub

🧬 Bioinfo-Bench: A Simple Benchmark Framework for LLM Bioinformatics Skills Evaluation

Introduction

We introduce Bioinfo-Bench, a novel yet straightforward benchmark framework suite crafted to assess the academic knowledge and data mining capabilities of foundational models in bioinformatics. Bioinfo-Bench systematically gathered data from three distinct perspectives: knowledge acquisition, knowledge analysis, and knowledge application, facilitating a comprehensive examination of LLMs in the field of Bioinformatics.

Bioinfo-Bench-qa

The questions are vary from 10 Bioinformatics domain, referring to the scope of Journal bioinformatics.

  • Genome analysis
  • Sequence analysis
  • Phylogenetics
  • Structural bioinformatics
  • Gene expression
  • Genetic and population analysis
  • Systems biology
  • Data and text mining
  • Databases and ontologies
  • Bioimage informatics

Bioinfo-Bench-seq

The system prompt are as follows:

You are given a nucleotide sequence that was originally a valid RNA. Exactly one error has been introduced randomly. Your task is to identify the type of error applied. Possible error types include:
- Reordering of existing bases
- Deletion of one or more bases
- Addition of an extra base (e.g., U)
- Replacement of one base with another
Choose the most appropriate error type from the options.