|Title||1||Provide as accurate and concise a description of the content of the article as possible.||This area is specific to a research article format and not appropriate for a database.|
|Abstract||2||Provide an accurate summary of the background, research objectives, including details of the species or strain of animal used, key methods, principal findings and conclusions of the study.||The information typically presented in a manuscript abstract is captured within the website and described within the remaining 18 items.|
|Background||3||a. Include sufficient scientific background (including relevant references to previous work) to understand the motivation and context for the study, and explain the experimental approach and rationale. |
b. Explain how and why the animal species and model being used can address the scientific objectives and, where appropriate, the study’s relevance to human biology.
|The IMPC webpage, Goals and Background, details the goals of the project and the relevance to human biology. In summary, the IMPC aims to systematically discover and ascribe biological function for every gene by generating a knockout mouse line for each protein-coding gene and identifying phenotypic changes between the knockout and control animals. Essential to this effort is to establish collaborative networks that work together to standardise protocols and data analysis.|
|Objectives||4||Clearly describe the primary and any secondary objectives of the study, or specific hypotheses being tested.||See Item 3|
|Ethical statement||5||Indicate the nature of the ethical review permissions, relevant licences (e.g. Animal [Scientific Procedures] Act 1986), and national or institutional guidelines for the care and use of animals, that cover the research.||The IMPC Consortium collects data from international member institutes who collect phenotyping data guided by their own ethical review panels, licenses, and accrediting bodies that reflect the national and/or geo-political constructs in which they operate. We have captured this data via an ethical and funding survey from each contributing institute.|
|Study design||6||For each experiment, give brief details of the study design including: |
a. The number of experimental and control groups.
b. Any steps taken to minimise the effects of subjective bias when allocating animals to treatment (e.g. randomisation procedure) and when assessing results (e.g. if done, describe who was blinded and when).
c. The experimental unit (e.g. a single animal, group or cage of animals). A time-line diagram or flow chart can be useful to illustrate how complex study designs were carried out.
|See Item 11|
|7||For each experiment and each experimental group, including controls, provide precise details of all procedures carried out. |
a. How (e.g. drug formulation and dose, site and route of administration, anaesthesia and analgesia used [including monitoring], surgical procedure, method of euthanasia). Provide details of any specialist equipment used,
b. When (e.g. time of day).
c. Where (e.g. home cage, laboratory, water maze).
d. Why (e.g. rationale for choice of specific anaesthetic, route of administration, drug dose used).
|The experimental procedures in IMPC are the backbone to the project, so very early in the project and pilots, a standardised data format and underlying database (IMPReSS) were developed. This allows us to capture and organise all the necessary information to define the protocols, ensure data reproducibility across centres, and enhance the data with the relevant meta-data for analysis. The IMPReSS database is based on a pipeline concept, which is a series of experimental protocols performed in order, a protocol that defines the method and the parameters and metadata to be measured. The information is organised such that each data point for a parameter of interest can be associated with the relevant procedural and mouse information.|
For each protocol the following are captured;
The purpose of the experiment.
The experimental design – number of animals, the age and sex tested.
Procedure – the protocol, which can also be called the standard operating procedure, followed.
Data QC – here information is presented on why a data point could be excluded. For example in the calorimetry screen the respiratory exchange ratio has to be between 0.7-1 as this reflects what Phenodcc
Parameters – the measured variables. Those that are marked as positive for “annotation” indicate variables of interest which will be processed through a statistical analysis pipeline.
Metadata – parameters that the area experts have determined as important for capture that could explain potential variation in the parameters of interest (e.g. equipment model). Those that are indicated as “required for analysis” are parameters that are used in data assembly for comparison.
IMPReSS, has been implemented in IMPC such that every animal (both mutant and control) is associated with a protocol which ensures that across the global project we can report what experiment was carried out on every animal and in what order. To provide trackability, IMPReSS also provides change history information on how the experimental definition has changed with time. IMPReSS thus provides the framework for not only transparency in the procedure and what data are required to be captured, but also stores information to facilitate subsequent analysis and thus is the backbone of the database and web portal. The development of this resource has required extensive collaboration with the institutes, area experts and those constructing the database.
|8||a. Provide details of the animals used, including species, strain, sex, developmental stage (e.g. mean or median age plus age range) and weight |
(e.g. mean or median weight plus weight range).
b. Provide further relevant information such as the source of animals, international strain nomenclature, genetic modification status (e.g. knock-out or transgenic), genotype, health/immune status, drug or test naïve, previous procedures, etc.
|The large scale of the project necessitated the development of an international tracking system (iMITS) which tracks the progress of the genetic modifications from all IMPC centres, starting at the planning stage for each centre, continuing through the microinjection of ES Cells, animal breeding and verification of their genotype, to the end of phenotype data capture. The coordination of production and phenotyping between all IMPC centres, and the ultimate repository of the genotypes and international strain nomenclature for each mutant allele produced by the IMPC is a critical component of the project output. Strain, mutation and production status information is fed forwards as needed to the various informatics tools needed to store, analyse and present the data.|
The IMPC informatics infrastructure has developed an automated method of data capture from all phenotyping centres. This method, is supported by a strict data standard that defines, in additional to the protocol from IMPReSS, data about each animal (Local ID, date of birth, strain, sex and centre). The volume of data on individual animals is vast, so we have worked with the community through extensive user testing to organise the information in as intuitive manner as possible. All of the IMPC data are available through both the web portal by a spreadsheet downloads from each graph or via programmatic access, and users can access this granularity of data on an animal level if required.
|Housing and |
|9||Provide details of: |
a. Housing (type of facility e.g. specific pathogen free [SPF]; type of cage or housing; bedding material; number of cage companions; tank shape and material etc. for fish).
b. Husbandry conditions (e.g. breeding programme, light/dark cycle, temperature, quality of water etc for fish, type of food, access to food and water, environmental enrichment).
c. Welfare-related assessments and interventions that were carried out prior to, during, or after the experiment.
|To capture housing and husbandry information, the international community came together and based on the requirement of the ARRIVE guidelines, the Gold Standard publication Checklist reporting Guidelines and the Genetically Altered (GA) Passport constructed a series of questions and answers which was used in the construction of a housing and husbandry survey.|
The phenotyping pipelines have been designed to ensure that there are no welfare related issues for a normal mouse, however incidental welfare issues may arise. These will be driven by both the environment and the genetic background of the mice, and commonly include runting, malocclusion and hydrocephalus. Animals presenting with these incidental welfare issues will be assessed on a case by case basis to determine if the issue can be managed through remedial care, such as wet mash on the cage floor for runting or clipping teeth in the case of malocclusions, or whether our ethical obligation to the animals and the scientific endpoints are better served by euthanizing the affected animal and providing a replacement. In addition to these incidental welfare concerns, a welfare issue could occur with genetically altered mice. Two standard complementary approaches are used within the community to identify and manage the potential systematic welfare issues that could arise. First, during the generation of the first homozygous progeny there are a series of assessments considering the basic dysmorphology to identify welfare concerns. If significant welfare concerns are raised; then the breeding strategy is modified to avoid the generation of homozygous mice and heterozygous mice are phenotyped instead. In addition to this early assessment, ongoing monitoring is carried out through the lifetime of the colony. Should the animals present a welfare concern either an intervention is made in the form of the Alternate Mouse Pipeline or, when applicable, remedial action is taken within husbandry (e.g. long water spouts or food on cage base) or within experimental procedures (e.g. an alternate anesthetic is used or tests are dropped). The Alternate Mouse Pipeline is an adapted phenotyping pipeline where a bespoke pipeline is run where the age and protocols are adapted to manage the welfare of the line. Currently the community is in the process of defining the Alternate Mouse Pipeline within IMPReSS that will in the future allow this data to be captured and disseminated. These two complementary approaches allow us to maximise the value of information extracted from the lines while minimising welfare issues. If the mice/cryopreserved sperm are requested by the community, the welfare concerns are documented in a report when ordered (e.g. to the Genetically Altered (GA) passport) from the mouse repositories (discussed further within item 19).
|Sample size||10||a. Specify the total number of animals used in each experiment, and the number of animals in each experimental group. |
b. Explain how the number of animals was arrived at. Provide details of any sample size calculation used.
c. Indicate the number of independent replications of each experiment, if relevant.
|As a high throughput project, the sample size is relatively low with a target number of knockout animals being processed of 14 (7 per sex). This number was arrived at after a community wide debate that involved statisticians, biologists and project managers to find the lowest number that would consume the least amount of resources while achieving the goal of detecting phenotype abnormalities in a strain. At times, practical issues might limit the number of animals it is possible to test such as viability issues or the difficulty in administering a test. As such, each time data are shown, the number of animals phenotyped per sex per genotype is listed with the graphical visualisation of the data.|
In a high throughput environment, replication of individual lines is not cost effective. Instead, multiple IMPC centres are generating and characterising the same six reference knockout lines that will present a group a wide range of phenotypes based on previously published research. The characterisation is on-going.
|Allocating animals |
|11||a. Give full details of how animals were allocated to experimental groups, including randomisation or matching if done. |
b. Describe the order in which the animals in the different experimental groups were treated and assessed.
|As a high throughput project that is looking to generate a hypothesis about gene function for all genes, the community is following a general design of having seven knockouts per sex that are compared to the control data, giving four groups of mice (wildtype males, wildtype females, knockout males and knockout females). In some cases (e.g. embryonic lethality for homozyogtes), heterozyogote mice are phenotyped and compared to wild-type mice. Within the analysis we consider the mouse as the experimental unit. Unlike most experiments, we cannot randomly allocate animals to experiment groups; rather we are relying on Mendelian inheritance providing the randomisation method. However, there are still many other aspects of the experiment where planning is necessary to avoid bias (e.g. order effects). Whilst the general approach and procedures are captured and tightly defined in IMPReSS, we identified that implementation (e.g. blinding) could vary significantly from institute to institute depending on local resources and priorities.|
As an international community, discussions on study design and implementation identified miscommunication as a significant problem. We found we could be using the same language but not actually meaning the same activity in practice. Addressing this issue required the development of a standardised language used to describe various aspects of the experimental design and resulted in the Mouse Experimental Design Ontology (MEDO). The practical reality of experimental design is that there is no perfect solution but instead transparency is needed to capture how experiments were implemented to allow users to independently assess the potential strengths and weaknesses in the design. A solution for one institute might not be appropriate for another and this can arise from the balance between removing risk and practical considerations where we need to follow the KISS design principle (keep it simple and straightforward). Examples of variation in implementation included how institute manage the potential bias from instrumentation and differences in blinding strategies with different levels of stringency. This variation demonstrates that experimental design, whether on a small or high throughput scale, doesn’t have an obvious simple solution but rather a spectrum of solutions.
As a community, we termed this study design information (in the context of a high throughput pipeline) as the ‘workflow’. The workflow data are currently captured in a survey format yearly using the MEDO ontology that captures the different pipelines an institute implements and is described in the “Experimental Design” protocol within IMPReSS. This level of transparency not only allows users to judge the quality of the experiments and potential risks themselves but also encourages good practice and communication at a detailed level between IMPC centres.
|12||Clearly define the primary and secondary experimental outcomes assessed (e.g. cell death, molecular markers, behavioural changes).||See Item 7|
|Statistical methods||13||a. Provide details of the statistical methods used for each analysis. |
b. Specify the unit of analysis for each dataset (e.g. single animal, group of animals, single neuron).
c. Describe any methods used to assess whether the data met the assumptions of the statistical approach.
|Ensuring that the appropriate statistical analysis is applied is a common problem in biology. In high throughput phenotyping, this is an area of active research [1, 2]. Developing an analysis pipeline for a resource with data from many sources is challenged by the number of variables, different data types, the data quantity for an institute, and variation in experimental workflow. An example of the variation in workflow includes the difference between data from the Institut Clinique de la Souris which is collected with a design of one batch of knockout mice with concurrent controls; while at the Wellcome Trust Sanger Institute the knockout and control mice are collected in multiple batches but not necessarily on the same day. The analysis implemented is further complicated by the requirement for the analysis to be completely unsupervised with no user intervention. As such, an analysis pipeline has to be robust, it must process the data consistently, and it cannot be fine-tuned for all possible scenarios.|
To address the analysis questions, the international community has come together to form an IMPC Statistical Technical Group to systematically address these issues and guide the research that is needed in this area. Consequently, we have implemented an analysis pipeline that applies the best statistical test based on the assay and the structure of the data (e.g. control method used). To ensure this analysis is transparent, we have developed a package of tools called PhenStat that uses the popular statistical language R. The PhenStat package is freely available from Bioconductor, and is versioned control as this is still active research in the area of data analysis. A statistics 101 document provides an explanation of the analysis.
On the mouse phenotype web portal, each data graph includes a statistical summary that includes p values and effect sizes with the ability for users to obtain more information about the statistical method implemented. As data can be analysed in many ways and each way has strengths and weaknesses, we have developed tools and procedures to enable users to download data for independent analysis by spreadsheet downloads, ftp access and automatic programmatic interfaces.
|Baseline data||14||For each experimental group, report relevant characteristics and health status of animals (e.g. weight, microbiological status, and drug or test naïve) prior to treatment or testing (this information can often be tabulated).||The challenge with the analysis is not only in selecting the most appropriate analysis platform but presenting all the results and the concomitant information in a user accessible way. Data outputs used to make genotype-phenotype associations are visualised graphically and augmented with p value as a measure of biological significance. Accompanying the graphs are summary measures for each group: including number of animals, and appropriate summaries for the data type (e.g. for continuous data the mean and standard deviation). The weight data can be separately accessed for a line of interest by looking at the weight curve data. |
Data are rarely excluded from the analysis. Exclusion can arise from two mechanisms. Firstly during data collection, data points can be quality control (QC) failed and an explanation provided using a standardised set of options as agreed by area experts. An example would be the explanation “Procedure Failed – Insufficient Sample – Blood sample”. Secondly, data are QC’d after upload from the institute to the database. Concerning data is investigated through collaboration between the inputting data centre and the Data Coordination Centre (DCC) using an internal QC web interface . This interface is designed to rapidly visualise the data and track concerns. This was essential when dealing which such large volumes of data. Data can only be QC failed from the dataset if clear technical reasons can be found for a measurement being an outlier. Reasons are provided and this is tracked within the database.
|Numbers analysed||15||a. Report the number of animals in each group included in each analysis. Report absolute numbers (e.g. 10/20, not 50%2). |
b. If any animals or data were not included in the analysis, explain why
|See item 14|
|Outcomes and |
|16||Report the results for each analysis carried out, with a measure of precision (e.g. standard error or confidence interval).||See item 14|
|Adverse events||17||a. Give details of all important adverse events in each experimental group. |
b. Describe any modifications to the experimental protocols made to reduce adverse events.
|As discussed in “Methods – Housing and husbandry (item 9)”, the pipelines have been designed such that a normal mouse would not be expected to present with adverse welfare indicators and systems are in place to minimise genotype specific welfare issues. Modifications required to ensure the welfare of a knockout line, are captured in the report issued with the mouse line if requested . Currently the community is in the process of defining the Alternate Mouse Pipeline within IMPReSS which is used when a knockout line has welfare issues such that standard pipeline is not applicable. In future, this new pipeline will allow this bespoke data to be captured and disseminated.|
|18||a. Interpret the results, taking into account the study objectives and hypotheses, current theory and other relevant studies in the literature. |
b. Comment on the study limitations including any potential sources of bias, any limitations of the animal model, and the imprecision associated with the results2.
c. Describe any implications of your experimental methods or findings for the replacement, refinement or reduction (the 3Rs) of the use of animals in research.
|The results are interpreted in an automated fashion following statistical analysis by the assignment of a mammalian phenotype ontology term when a significance threshold agreed upon by the community is reached. A mammalian phenotype term (MP term) is a standardised ontology for describing a phenotype developed by the Mouse Genome Informatics group. For example the MP term MP:0005634 – decreased circulating sodium level – is defined as less than the normal concentration of this ion in the blood and it is assigned to mutant mouse strains when blood sodium values for members of this strain are significantly lowered when compared to controls. Phenotype assignment can also occur for a particular sex of a mutant strain when the effect is only observed in one sex. The use of the standardised ontology is a critical step to allow comparison across studies and species.|
The ARRIVE guidelines asks for comments on the limitations of the study, including any potential sources of bias and imprecision associated with the results (item 18b). This is an area of active discussion amongst the IMPC Statistical Technical Group with ongoing research into methods to address this. However, a database of this size is limited by the requirement that analyses need to be automated and reliable.
|19||Comment on whether, and how, the findings of this study are likely to translate to other species or systems, including any relevance to human biology.||Mouse repositories play an important role in the generalizability of results by allowing researchers to perform follow up studies on genetically identical animals as those used in previous analysis. All strains generated as part of the IMPC project are available to the research community via the established mouse repositories Knockout Mouse Repository and INFRAFRONTIER. These repositories include details about allele structure, genetic background, pathogen exclusion list and any potential issues in husbandry and welfare that may result from carried mutations. This information increases the likelihood of reproducing previous results, the lack of which is seen as a detriment to translating mouse research studies to human biology .|
Many animal experiments study only one sex, typically the males, to avoid potential issues with oestrogen cycles. There has been growing concern over the gender imbalance in biomedical research [4,5,6] and this had led to a policy shift within the US National Institutes of Health such that they now require all grant applicants to report their plans for balancing male and female animals in preclinical studies. Fortunately, IMPC members decided at the start of the project that phenotype data should be collected from both sexes and this design supports the generalizability of the findings. In addition, our continuous analysis pipeline is designed to detect sexual dimorphism and we associated a tag to the MP terms to classify the effect observed.
Mouse knockout (or null) mutations lead to the absence of the gene product and is recognized as a standard approach in all model organisms for assessing gene function. The comprehensive phenotypic analysis of mouse knockouts being carried out by the IMPC provides an important baseline of mammalian gene function. However, not all aspects of gene function will be revealed by the generation of a knockout mutation. For example, gain of function effects that might arise from the introduction of a specific amino acid change in a protein would not be uncovered. Indeed, many common variants in the human population represent alleles with a variety of effects, from partial loss of function (hypomorphs) to gain of function (neomorphs). Thus the phenotypic outcomes of these types of allelic effects in humans will not be directly modelled by knockout mutations. Nevertheless, many Mendelian disorders, including rare diseases, in the human population will be caused by severe loss of function effects for which the data generated from mouse knockout mutations will be highly informative. Moreover, recent studies have found a large number of associations between the phenotypes of Mendelian and complex diseases that link complex disorders to a unique spectrum or “code” of Mendelian loci. In addition, common variants associated with complex disease are enriched in those Mendelian loci revealed by this code. This underscores the wider utility of examining loss of function alleles in the mouse. We can expect in some cases that the pleiotropic effects uncovered from mouse knockout mutations will both extend and validate information on the potentially diverse phenotypes arising from variant alleles in the human population.
The main goal of the IMPC is therefore to generate a complete functional catalogue of the mouse genome containing knowledge that can be translated to other species, especially human. Given the mouse’s status as the model organism for understanding human disease and basic human biology
|Funding||20||List all funding sources (including grant number) and the role of the funder(s) in the study.||We have captured this data via an ethical and funding survey from each contributing institute.|