Skip to main content

One Health VBD Hub Training 2026

4 Understanding Data Curation and Standards

4.1 Overview

So far, we have learnt how to search for and retrieve data using VBD Hub resources. In this session, we will look at the other side of that process and consider how datasets are curated and standardised before they reach the Hub. We will discuss publishing decision frameworks to help decide where and how a dataset is shared, how FAIR data principles guide the whole process, and you will have the opportunity to prepare a dataset for VecDyn yourself. Understanding data curation and standards will help to interpret the datasets you retrieve from existing databases, and prepare your own data for sharing.

4.1.1 Learning Objectives

By the end of this workshop, you should be able to:

  1. Prepare a dataset for VecDyn using the VecDyn column definitions.
  2. Apply the FAIR data principles to assess and improve a dataset.
  3. Explain how curation and standards shape the data you retrieve with Hub Search and ohvbd.
  4. Appropriately cite datasets, repositories and resources.

4.2 Identifying Data Types

In the pre-work, you were introduced to the different types of data the VBD Hub works with, and the standard operating procedures (SOPs) used to curate each type. Identifying the data type is the first step in curation, because it determines where a dataset is stored and how it should be formatted. Let’s start with a quick recap.

Task: Identifying Data Types

For each example, decide which type of data it is:

  1. Weekly counts of Culex pipiens caught in light traps at five sites over two years.
  2. A record of Aedes albopictus observed in a garden in Kent in August 2023.
  3. Larval development time of Anopheles gambiae at five different temperatures.
  4. Monthly reported human West Nile virus cases by region.
  5. Mass spectrometry analysis of proteins in mosquito salivary glands.
  6. Blood-meal analysis showing which host species Culex pipiens fed on.
  7. Museum specimen records of Ixodes ricinus with collection dates and locations.

4.3 Publishing Decision Framework

To ensure datasets are shared in the most appropriate way, the VBD Hub uses a publishing decision framework.

This framework guides three key decisions:

  • What type of data is being shared
  • Which database it should be stored in
  • How it should be formatted

Once you have identified the data type, you can map this to the recommended repository:

  • Omics data are typically stored in genomic and proteomic repositories such as GenBank or ProteomeXchange.
  • Abundance data are directed to VecDyn.
  • Trait data are stored in VecTraits.
  • Occurrence data are shared through platforms like GBIF.
  • Epidemiological or mixed datasets may be hosted directly by the VBD Hub.

This ensures that each dataset is stored in a system best suited to its structure and use.

Example: Preparing a Dataset for VecDyn

Imagine we have collected mosquito abundance data weekly using light traps at several sampling locations.

Our raw dataset might look something like this:

  • A column called Date with mixed formats
  • A column called site with inconsistent naming
  • A column called Count with no explanation of how counts were collected.

We then want to prepare this dataset for VecDyn.

Instead of choosing column names ourselves, we use the VecDyn column definitions resource. This provides a standardised list of column names, along with clear definitions of what each column name means and how it should be used.

For example:

  • If we have a column describing when the sampling took place, the table specifies the column names sample_start_date and sample_end_date.
  • If we want to record each sampling location, we would use sample_location.
  • If we are recording the species, we can use standard taxonomy in species and genus columns.
  • Our abundance dataset contains count data, which we would record in a column called sample_value.

The VecDyn definitions table is accompanied by additional information that guides how the data should be structured:

  • Notes - provides further explanation, helping to clarify how the column should be used and what the values represent.
  • Is Required - indicates whether a column must be included in the dataset to be accepted and usable within the VecDyn database.
  • Data Format - specifies how data values should be recorded. For example, dates must follow a consistent ISO format.
  • Restrictions - provides rules about what values are allowed. This might include controlled vocabularies or limits on acceptable entries.

Given these details, we would also want to check how our data is formatted, for example, our start and end date must be in ISO format. We would also need to add any required details, such as a sampling_method column to state the trap type.

By following these definitions, we are not just organising our dataset, we are aligning it with a shared formatting standard. This is what allows tools like the VBD Hub Search and the ohvbd package to work with multiple datasets in a consistent way - they can rely on the same column names, formats, and definitions across all data.

4.4 FAIR Data Principles

An important concept guiding this entire data sharing process is the use of FAIR data principles.

FAIR stands for Findable, Accessible, Interoperable, Reusable:

  • Findable - datasets are assigned metadata and identifiers so they can be discovered through search tools.
  • Accessible - data can be retrieved using standard protocols, with clear information about access conditions.
  • Interoperable - datasets use standard formats, vocabularies, and structures so they can be integrated with other data.
  • Reusable - datasets include sufficient documentation, metadata, and licensing so they can be used in future research.

The VBD Hub supports these principles by ensuring datasets are well-documented, consistently structured, and linked to appropriate repositories, so that these datasets can be discovered, accessed and reused openly and effectively.

Task: Preparing a Dataset for VecDyn

You’ve been given the following raw data and short Methods paragraph. Working in small groups, use the VecDyn column definitions resource to prepare this dataset for upload to VecDyn.

Here are some prompts to help you:

  • Which VecDyn column should each existing column map to?
  • What needs to change in the values themselves?
  • How might you work out the start and end dates where these are missing?
  • What required information is missing?
  • How does the metadata from this dataset support each of the FAIR principles?

Methods

Adult mosquitoes were collected using Mosquito Magnet, BG-Pro and BG-Sentinel 2 traps, and larvae and pupae were collected by dipping. Surveillance ran from 1st June to 31st October 2024. Catches were pooled and reported by sampling week. For each record, the Date is the first day of a 7-day sampling week, and the count is the number of Culex modestus caught at that site during that week. At some sites, Culex modestus was not found at any point during the survey. For these sites, there is a single record with a count of 0. This record covers the whole survey period, and its Date is the first day of the survey. All specimens were identified morphologically.

(This dataset and Methods has been edited for this task. The original dataset is from Abbott et al. (2025) “Evidence of much wider distribution of the potential West Nile virus vector, Culex modestus, in the UK”, Parasites & Vectors 18: 482, doi:10.1186/s13071-025-06936-3.

4.5 Why Use Curated Data and Standards?

Understanding how data is curated and structured also helps when using tools like the Hub Search or the ohvbd package.

For example, when you retrieve data, you may notice:

  • Standardised column names
  • Consistent date formats
  • Structured metadata

These features are a direct result of the data curation process.

Recognising this can help you interpret datasets more effectively and identify any limitations or inconsistencies. For example, you might notice multiple columns for species names based on different databases, which you can mitigate by merging.

These standards aren’t only useful when using other people’s data. Applying them to your own data from the start brings the same benefits:

  • Your datasets are structured the same way across projects, field seasons, and team members.
  • Your data can be easily combined with other datasets without extensive formatting.
  • Recording data in a standard format from the outset means you spend less time cleaning and wrangling later on.
  • Improves the reproducibility of your work, especially for publications.
  • If your data already follows VecDyn or VecTraits standards, it is ready to submit when you publish, so your work can be found, reused and cited by others.

4.6 From the Data Curator

Sarah is the VBD Hub Data Curator. Her job involves finding the data we all want to see, taking a deep dive into the ongoings of the data, reshaping it into something that can be reused over and over again and building relationships within the community to create trust in the Hub, trust to share their data in a way that gives them full control and let’s the world see their research.

We’ve asked Sarah some questions about the data curation process and submitting data to the VBD Hub.

What are the most common problems you see in datasets submitted to the Hub?

“1. Missing data - When planning your experiment, right at the start, think about what variables you need to use to make this reusable. So we often have:

  • Missing dates
  • Missing species identification methods
  • Missing locations
  • Missing trap types
  • Missing information on how long the traps are left out for

  1. Poor data dictionaries - When a curator asks the provider what the data means and they too are confused. This is often caused by instructions lost in communication between team members, all recording things in slightly different ways.

  2. Problems with zeroes - What does your 0 mean? Does it mean a trap failed, does it mean a trap was put out and none of that mosquito species were collected, why were those mosquito species put in your dataset anyway?

  3. Dates - They can be differently interpreted from where you are in the world so let’s please use YYYY-MM-DD.

  4. Data in wide format (not long format) - which requires more wrangling!”

What’s one thing researchers could do at the data collection stage that would make curation much easier?

“1. Think about reusability, think about if you wanted to use your data in the future for a different analysis. Have you collected enough information and in an understandable way that you could pick it up in 3 years time and know what it all means?

  1. Standardise your data collection sheets! Give all your team the same data collection templates.

  2. Look at a data repository you want to deposit in and check you have all the required fields before you start collecting data.”

What happens to a dataset after it’s submitted?

“This depends on what data you submit to the Hub. In general what will happen is that I will run checks on the data to check all the required fields for that data type are in the dataset. I will check if I have data dictionaries/methods/publications that will help me to understand the data and then I will reshape the data into the format required for submission to the individual repositories. Anything that is unclear or if I notice data duplications or missing location info etc I will contact the provider. The more info you provide, the less I have to bother you!”

How should people get in touch if they want to share their data?

“There are a few ways to get hold of the Hub team but if you want to go direct to source, please email me at . I don’t bite, I don’t mind how messy your data is, trust me I have seen it all and that is what I am here for! To help you future proof your own work. To take away some of that burden.

I think it is actually always really helpful to reach out to me at the very beginning of your study! I can help check you are collecting all the right data to make your data FAIR. Having your data in a place for all to see leads to some awesome future collaboration opportunities when someone discovers your data and wants to reach out to you! So, we require you to share your contact information with us on the dataset so you can be reached directly.”

4.7 Citing Curated Datasets

When using datasets from the VBD Hub, it is important to consider data provenance and attribution. You should:

  • Check the dataset metadata for authorship and source.
  • Identify the original database.
  • Follow licensing and usage conditions.
  • Cite both the dataset and the source repository.

Proper citation:

  • Gives credit to data contributors
  • Supports reproducibility
  • Ensures transparency in your research

Tip: Remember to cite the tools you use to retrieve data as well. If you used ohvbd, run citation(“ohvbd) in R to get the recommended citation.

4.8 Key Takeaways

  • Identifying the data type is the first step in curation, because it determines where a dataset is stored and how it is formatted.
  • Standard column definitions, formats and vocabularies allow datasets from different sources to be combined, which is what makes tools like the Hub Search and ohvbd possible.
  • The FAIR data principles guide curation so that data can be found, accessed, integrated and reused.
  • Applying data curation standards to your own data from the start saves time later and makes your data easier to share, reuse and cite.
  • Always cite the datasets, repositories and tools you use.