Login to MyKarger

New to MyKarger? Click here to sign up.



Login with Facebook

Forgot your password?

Authors, Editors, Reviewers

For Manuscript Submission, Check or Review Login please go to Submission Websites List.

Submission Websites List

Institutional Login
(Shibboleth or Open Athens)

For the academic login, please select your country in the dropdown list. You will be redirected to verify your credentials.

Original Paper

Free Access

Testing an Online, Dynamic Consent Portal for Large Population Biobank Research

Thiel D.B.a · Platt J.a, b · Platt T.a · King S.B.a · Fisher N.a · Shelton R.d · Kardia S.L.R.a, c

Author affiliations

aLife Sciences and Society Program, and Departments of bHealth Management and Policy and cEpidemiology, University of Michigan School of Public Health, Ann Arbor, Mich., and dPrivate Access Inc., Irvine, Calif., USA

Corresponding Author

Daniel B. Thiel

Life Sciences and Society Program, University of Michigan School of Public Health

1415 Washington Heights, Suite 4605

Ann Arbor, MI 48109-2029 (USA)

E-Mail dbthiel@umich.edu

Related Articles for ""

Public Health Genomics 2015;18:26-39

Abstract

Background: Michigan's BioTrust for Health, a public health research biobank comprised of residual dried bloodspot (DBS) cards from newborn screening contains over 4 million samples collected without written consent. Participant-centric initiatives are IT tools that hold great promise to address the consent challenges in biobank research. Methods: Working with Private Access™ Inc., a pioneer in patient-centric web solutions, we created and pilot tested a dynamic informed consent simulation, paired with an educational website, focusing on consent for research utilizing DBSs in Michigan's BioTrust for Health. Results: Out of 187 pilot testers recruited in 2 groups, 137 completed the consent simulation and exit survey. Over 50% indicated their willingness to set up an account if the simulation went live and to recommend it to others. Participants raised concerns about the process of identity verification and appeared to have little experience with sharing health information online. Conclusions: Applying online, dynamic approaches to address the consent challenges raised by biobanks with legacy sample collections should be explored, given the positive reaction to our pilot test and the strong preference for active consent. Balancing security and privacy with accessibility and ease of use will continue to be a challenge.

© 2014 S. Karger AG, Basel


Introduction

Collections of biospecimens in large population biobanks are increasingly valuable for health and scientific research. When collection has preceded informed consent - as is the case for retrospective biobanks - significant questions related to informed consent follow [1,2,3,4,5,6,7]. In some cases, de-identification has been proposed as a mechanism for reducing risk such that written, informed consent is not necessary. Opt-out procedures maximize ongoing participation but face ethical questions about their ability to allow participants autonomy if participants are unaware of their participation [8,9,10]. Even when consent is obtained, retrospective biobanks face the challenge - common to all types of biobanks - of whether consenting participants can be truly informed in the process of opting into unknown, future research [8,9,10,11].

Michigan is home to a state biobank comprising residual dried bloodspots (DBSs) left over from newborn screening, including some 4 million ‘retrospective' DBSs collected from a generation of Michiganders before written consent policies were in place [12,13]. The incorporation of Michigan's BioTrust for Health (BioTrust) in 2010 was significant in the history of newborn blood spot biobanking in the US, as it was a first-of-its-kind effort to bring the research ‘goldmine' of ∼4 million DBS samples to the research community to promote and stimulate new research [13,14]. The grandfathered collection of DBSs, collected between July 1984 and May 2010 without written consent were combined in this biobank with new DBSs, which are now added to the research pool only with written parental consent [13]. Thus, the biobank has a dual consent policy for secondary research uses of DBSs, an opt-in, broad, parental-proxy consent for new spots and an opt-out option for those who did not have the opt-in process available to them. This raises the issues noted above to ‘opt out' approaches.

In light of the consent challenges facing the BioTrust and other retrospective biobanks, online and dynamic approaches to solving the problem of consent for biobanking are gaining attention and adherents [15,16,17,18,19,20]. We simulated an online, dynamic tool designed to simultaneously educate Michiganders and recorded tiered consent preferences for participation in the BioTrust. This study presents user experiences with the online tool that was designed to be easy to implement, publicly accessible, and a tool that meets the ethical goals of informed consent.

Our efforts build upon the call for participant-centric initiatives (PCIs), IT tools designed to place patients and research participants at the center of health and research decision making [17,21,22,23]. These initiatives, often referred to as ‘dynamic consent' models, seek to implement ‘new methods for consent and for exercising choice over the use of samples and information in response to changing research needs while not hampering research with burdensome practices' [17,24]. Even though the model appears promising on many fronts, efforts to implement dynamic consent for biobanking have been limited. The Ensuring Consent and Revocation Project and the Oxford Radcliffe Biobank in the UK have undertaken such a project and are in the testing phase of a dynamic consent tool [9,17,20,23]. Our PCI approach to consent for the BioTrust represents a first for DBS biobanking in the US.

In a rapidly advancing health information economy, online tools could be used to establish an access point for public education and communication regarding the BioTrust, particularly among parents and adult ‘retrospective' participants [25]. Hypothetically, donors (or parents of minors) could use an online consent portal to register consent or dissent for DBS research use. With an asynchronous consent process, parents and participants who never had an opportunity to consent at the time of collection could register their preferences, at their convenience and outside of normal interactions in a clinic, hospital or health department. Furthermore, an online platform would be flexible enough to enable participants to personalize options (tiered consent) and choose to amend consent preferences on an ongoing basis [26]. Ideally, such a system would be incorporated into the existing data architecture of the biobank. For example, individual consent preferences could become a searchable data ‘tag' associated with each sample's digital record. In the long term, individuals whose parents initially provided consent to participate in the BioTrust would also have a point of access upon reaching the age of majority. Individuals who prefer a more active role in biobank participation could be given greater opportunity to engage as partners in the research enterprise, and those who prefer a one-time interaction would be afforded a simple and convenient way to register their preference once and for all [27].

In partnership with Private Access™ Inc., we created and tested an education portal and consent simulation for a large population biobank. Private Access is recognized in this field as innovating ‘matchmaking' between rare disease patients and researchers [23]. Shelton [16] presents the perspective of the CEO of Private Access on the significance of this particular project and on the broader field of patient/participant-centered approaches to privacy and consent in health research, arguing that properly configured online consent will be a technological solution that respects individual's privacy wishes and simultaneously facilitates and strengthens the research enterprise.

Michigan's approach to, and experience with, implementing DBS biobanking has broad implications [28]. A third of US states retain DBSs for long-term use [29], and commonalities in the structure and design of the BioTrust with large population biobanks in general make the Michigan experience relevant and instructive to the broader biobanking and newborn screening communities [12,13,25,30,31]. Public opinion research demonstrates that individuals prefer to give consent for research use of their de-identified biospecimens [1,32,33,34,35] and, although the fate of proposed changes to the Common Rule are uncertain, the call for broad, written consent for research on de-identified biospecimens collected prospectively was a key element in the proposal [36]. In light of its practice of gathering broad use consent starting in 2010, the BioTrust offers a kind of test case for the proposed approach and will likely yield important insights in the coming years on the soundness of this approach. Further, with the majority of the BioTrust's collection coming from grandfathered DBSs, it will be important to track how the ‘opt out' approach that governs the disposition of these spots fares as an approach to consent in the years to come.

In what follows, we present data collected from our pilot test of this online, dynamic consent simulation and education platform. Specifically, this paper focuses on measures of user experience and satisfaction with the simulation and predictors (both demographic and user experience based) of the likelihood of setting up an account if it were a live system.

Methods

Working with Private Access, we designed, developed and pilot tested a simulated online consent portal for the Michigan BioTrust for Health, with a particular focus on consenting ‘donors' and parental proxies to the retrospective collection of DBSs. Before recruiting participants, we launched an educational website, explorebiobanking.org, which served as the starting point for pilot testers who were instructed to browse educational content related to the BioTrust and prompted to consent to participate in the Private Access study. Key educational components included short animated videos and an FAQ page (Explorebiobanking.org was deactivated in 2012 and has since been replaced by a more streamlined site, mybloodspot.org, which hosts similar content, including the animations used for the pilot testing.)

Recruitment: Two Pilot Test Groups

Research participants were recruited to pilot test the Private Access consent portal in 2 groups. Group 1 was comprised of Michigan citizens recruited with the help of 6 community-based partners who had previously collaborated in our efforts to engage Michiganders about the BioTrust by co-hosting community meetings on the topic in Flint, Grand Rapids, Jackson, Detroit, Dearborn, and Petoskey (Northern Mich., USA) [37].

To aid in recruitment, community partners were given promotional posters and trifold fliers along with a stack of postcards containing instructions for at-home completion of the simulation and a unique identifying code so that we could track the recruitment rates around the state. Recruits were asked to complete the process within a 2-month time frame. A total of 385 recruitment postcards were distributed among the community partner organizations (table 1). Due to a lower-than-expected participation rate, we decided to focus a second recruitment effort on a group that, we hypothesized, would be particularly willing to utilize an online consent portal, namely college-aged students (∼18-22 years). Additionally, this group would likely comprise a number of individuals who themselves were ‘donors' to the BioTrust collection of DBSs.

Table 1

Pilot test enrollment

http://www.karger.com/WebMaterial/ShowPic/140315

Group 2 was, therefore, comprised of students from the University of Michigan (UM), Ann Arbor campus, who were informed of the study via on-campus flyers and in-person recruitment to complete the simulation in computer labs on campus.

All participants completed an Institutional Review Board-approved online consent and were compensated with USD 25 for their time. Table 1 presents a summary of the recruitment data for each group and table 2 summarizes demographic characteristics of each group. In addition, table 2 presents summary demographic information for participants who attended the 10 community meetings we held in the state of Michigan on this issue for the sake of comparison.

Table 2

Community meeting and pilot test participant demographics

http://www.karger.com/WebMaterial/ShowPic/140314

Private Access Features

Account Setup, Select-a-Guide, User Preferences Matrix

Figure 1 illustrates the step-by-step process through the simulation. Participants were instructed to create a user profile that included giving basic information (e.g. name, contact information, etc.), creating a user name, password, site-key, and establishing 3 security questions. After the account setup, users were shown a sample user agreement and given the opportunity to review basic information about the BioTrust through links to content from explorebiobanking.org.

Fig. 1

Private Access consent simulation. a Steps 1-4. b Steps 5-8.

http://www.karger.com/WebMaterial/ShowPic/140311

As users proceeded through the stages of the simulation, they were prompted to either input data or shown prompts where data would be required if it were a live system. In cases where users were shown a hypothetical data request or privacy enhancement, they were asked to share whether or not they would be comfortable providing the information requested; open comment fields in these instances enabled us to collect feedback on these specific elements of the simulation.

After the initial setup, users reached the main activity of setting tiered consent and contact preferences. Users had the option to either set their consent and contact preferences manually or to consider and, if they wished, follow advice from one of 2 available ‘guides,' real-life public health experts whose brief biographies and photographs ‘personalized' their relationship to public health and biobanking. The guides offered preset selections for the 2 × 4 preference matrix, comprised of 2 key questions and 4 categories of researchers, with settings corresponding to tiered privacy concerns. Additionally, users could toggle among 3 completely preset matrices according to whether the user had greater, moderate or lower privacy concerns using color-coded tabs. Figure 2 presents an annotated screen shot of this central element of the consent simulation. After the matrix choices were set, participants were prompted to confirm their selections before continuing on to an identity (ID) verification step.

Fig. 2

Private Access consent preferences matrix.

http://www.karger.com/WebMaterial/ShowPic/140310

ID Verification

One of the challenges associated with online systems for managing personal health data is verifying that the individual who is using the online tool to manage health information is in fact the individual they claim to be. In developing our simulation, we were cognizant that the state of Michigan created the requirement that individuals verify their identity with a copy of their state-issued ID in order to successfully opt out of the BioTrust. We worked with Private Access to determine what appropriate and available mechanisms for online ID verification could be incorporated into the simulation. In addition to the creation of a user name and password, a 2-factor authentication was added that drew on publicly available data sources (banking history, mortgage records and state of issuance of social security number). Each user was presented with 3 questions about their banking activity and the state of origin of their social security number. In a live system, if the user answered all 3 questions correctly, then they would be deemed to be the individual they claimed to be, and their consent preferences would thus be validated.

Since our users were going through the process as a simulation, we only showed them a sample ID verification page, after they completed the consent matrix, and asked: ‘Would you answer these questions or questions like these?' Participants who indicated ‘no' were further asked: ‘If no, how would you suggest verifying identification?'

Final Steps

Upon completion of the ID verification, users were given the option of testing out a family health history creation tool (not discussed in this paper) and then directed to the exit survey (hosted on Survey Monkey). Thus, 3 main data sources were available for analysis from each group: (1) a pre-survey, (2) preferences and information entered during the consent simulation, and (3) an exit survey.

Analytical Approach

Descriptive statistics were generated for 6 demographic variables: gender, race/ethnicity, age, altruism, and self-rated health. Because altruistic values are highly prevalent among blood donors [38], we employed a measure of altruism defined as whether the participant self-identified as a blood donor. This measure replicated a question used to gauge respondent's altruistic beliefs and practices in a study evaluating veterans' attitudes toward participating in biobank research [39] and in a study evaluating public attitudes about a proposed large-scale national cohort study of genes and environment [40]. χ2 and Fisher's exact tests were then used to ascertain significant differences in demographic characteristics between the community group and the UM student group (table 2). We similarly compared demographic characteristics of those who completed the simulation versus those who did not (table 2).

We developed a 2-step logistic regression model to identify predictors of whether an individual would be likely to set up an account based on responses to the exit survey question: ‘If this were real (live) instead of a simulation, would you set up a Private Access account to manage permissions for use of your or your child's bloodspots?' using demographic variables, user experience variables and participant preferences regarding consent and contact. User experience variables included: overall experience, impressions of length of time to completion and ease of navigation, likelihood to recommend the service to others, reasons for not using an online personal health record service (e.g. Google Health), preference for privacy settings (i.e. set by guide or customized), and willingness to respond to ID verification questions. Consent and contact preferences were measured by responses to 2 questions about whether an individual would like to be asked each time their DBS would be used in research and whether parents should be able to decline having their baby's blood stored for use in research. We first conducted univariable analysis using each of the predictors that we hypothesized would be associated with the likelihood a participant would set up a Private Access account. Based on the results of the univariable logistic regression model, we used α <0.10 as the criterion for inclusion and exclusion in our multivariable modeling of this outcome.

Results

Results from the pilot tests are presented in 2 main sections: Recruitment and User Experience.

Recruitment: Demographic Characteristics of the Groups

Table 2 summarizes the demographic characteristics of each group and further shows the participant demographics from the first research arm of this project, namely a series of 10 community meetings (n = 393) held in the state of Michigan [37]. As compared with the community meetings, there was a notable improvement in the gender balance among pilot testers in both groups. Community group demographics were more closely aligned with the distribution of participants in our earlier community meetings across race/ethnicity, age and education variables than the student group.

A statistically significant difference was detected between the 2 pilot test groups along several demographic variables: gender (p = 0.03), race/ethnicity (p < 0.001), education (p < 0.001), age (p < 0.001) and an indicator of altruism, (viz. participant has donated blood to the Red Cross for medical use) (p = 0.039).

User Experience

Overall Participant Experience

Pilot testers' overall experience with Private Access was mostly positive or neutral, with relatively small numbers of participants indicating negative experiences in both groups (table 3). Among community group participants, 64.3% indicated a ‘somewhat' or ‘very' positive experience with the site overall, whereas 68.7% of the student group held this view.

Table 3

Summary of participant experiences and preferences by cohort affiliation

http://www.karger.com/WebMaterial/ShowPic/140313

Time to Completion

A significant difference between the groups emerged with their impressions of the time it took to complete the simulation (p = 0.003). While 42.9% of community group participants indicated that it was less time consuming than they had anticipated, only 17.4% of the students found this to be the case. At the other end of the spectrum, 20.9% of the student participants found that the simulation took too long, while only 5.4% of community group participants registered this view. Well over a third of each group (35.7 and 44.2%) indicated that the process took about the right amount of time.

Ease of Navigation

Users in both groups overwhelmingly indicated that the navigation through the simulation was either ‘fairly' or ‘very' easy: 82.1% (community groups) and 89.5% (student group) (table 3). Differences between the groups were not statistically significant for this measure of user experience.

Recommend versus Use

The willingness to recommend Private Access to others exceeded personal interest in setting up an account for both groups, but group affiliation was not significant for either question. The student group was almost evenly divided on the likelihood that they would set up an account, with 52.3% of the participants answering ‘yes' and an equally divided dissenting opinion between 22.1% saying ‘no' and 25.6% selecting ‘not sure' (table 3). Yet, for this group, 77.9% were either ‘somewhat' or ‘very' likely to recommend Private Access to others.

Community group participants were slightly more enthusiastic about creating an account for themselves, with 58.9% saying ‘yes', only 16.1% selecting ‘no' and 25% registering a ‘not sure.' This group also demonstrated greater enthusiasm for recommending the service to others, with 82.2% of participants indicating that they would be ‘somewhat' or ‘very' likely to recommend it.

Comfort/Experience with Online Health Information Services

A mere 3.6% of community participants and 2.3% of UM student respondents indicated that they had experience using online systems to manage and access their health information. A follow-up question queried reasons why participants did not utilize such a service. Half of community participants indicated that they would not want their medical record on the web and 40% cited a lack of understanding of how such a service would work. Fewer participants in the student group indicated concern about having their medical record on the web (30.6%), though a greater number (51.8%) indicated that they did not use a web-based service because they ‘did not know how this would work' (table 3).

User Actions

Guides and Prompts

Community group participants were far more inclined than students to avail themselves of preset options suggested by the guides (63 vs. 9.5%) (table 3). A full 90.5% of the student group opted to set custom preferences rather than follow the advice of the guides. This difference between groups was significant (p < 0.001).

ID Verification

Pilot testers from the community groups were generally more comfortable with the idea of answering ID verification questions, with 82% saying ‘yes', while the students registered more skepticism with only 59.8% saying ‘yes' to such an approach to ID verification. This difference in group preference proved significant upon further analysis (p = 0.005) (table 3).

As one of the few opportunities for sharing qualitative input, the responses of our testers to the open-ended follow-up question are noteworthy. Out of 52 comments, 18 indicated some form of critique of the ID verification process. Typical replies focused on the personal and private nature of the information being asked, as in ‘Getting very personal. I don't like talking about anything that has to do with my ssn [social security number] online' and ‘I don't know [if I would answer the ID verification questions] but I would not answer questions about my banking history willingly on a website that has nothing to do with that', or ‘the bank account and credit profile questions seem irrelevant and suspicious'. In 2 additional open-ended questions on the exit survey, 7 respondents came back to their concerns about the ID verification process, for example ‘I didn't like the ssn parts. Difficult and unnerving', and ‘the site LOOKS secure but the questions asked are very very private'. The latter quote captures the ambivalence that a number of participants expressed about their experience with the site. Many noted that it felt secure and professional, but they were still concerned about sharing such personal information online.

Likelihood of Future Use

Over half of the pilot testers (58.9% of participants in the community group and 52.3% of UM student participants) indicated that they would be likely to set up an account (p = 0.638) (table 3).

Logistic Regression

Based on the results of the univariable logistic regression models, we found that none of the demographic variables proved significant (p > 0.10) (table 4). Of the 7 user experience factors analyzed, we found 6 of them to be significantly (p < 0.10) associated with the likelihood of setting up an account: (1) the participants' overall experience, (2) the assessment of the length of time it took to complete the simulation, (3) the impression of ease of navigation, (4) the likelihood to recommend the service to others, (5) the expressed preference to not have one's medical record on the web, and (6) the reaction to the ID verification process. One of the 2 consent and contact preferences variables was significant in the univariable analysis: participants' reactions to the question of whether they would like to be asked each time their DBSs would be used in a research project (OR = 2.29, p = 0.027).

Table 4

Predictors of whether participants would set up an account (n = 137)

http://www.karger.com/WebMaterial/ShowPic/140312

Based on an inclusion/exclusion criterion of α < 0.10, our 2-step multivariable regression models indicate that 4 factors stand out as particularly salient as to whether an individual is likely to set up an account. Individuals who indicated a positive overall experience were more likely than those who indicated that their experience was somewhat or very negative to set up an account (OR = 9.23, p = 0.052), and individuals that were somewhat or very likely to recommend the simulation to others (OR = 5.20, p = 0.010) were also more likely to set up an account. A positive response to the ID verification step (indicating comfort with the proposed verification process) remained a significantly associated variable (OR = 2.34, p = 0.096), as did the desire to be asked each time one's DBS would be used for research (OR = 2.20, p = 0.078).

Discussion

Overall User Experience

Overall, participants who completed the process indicated a positive experience with the Private Access informed consent simulation. The data indicate that participants' decisions about whether or not to participate or recommend the service to others were not negatively impacted by basic design issues or simulation length. The 20% of students who found that the simulation took too long indicates that the design and ease of use expectations are greater for younger users.

Participants' experience, and comfort level, with online systems for health records and information was quite low among both of our groups. As this was a convenience sample, we cannot generalize these rates; however, the reasons that were given for not using an online portal may be instructive to those considering implementation of online education and consent for a biobank. The large number of participants from each group who would not want their health information online indicates minimally that measures need to be taken to enhance people's sense of comfort with online health information systems, though this is less so for the student-aged population.

Recruitment

For the purposes of our pilot testing, recruitment via campuses and community groups was adequate for meeting our aims. Monetary incentives and, among community groups, help from trusted community partners to distribute recruitment postcards helped motivate participation. On a statewide scale, a sustained effort to inform the public and promote the tool would be necessary to garner widespread participation. Relatively modest investments in outreach via social media outlets (e.g. Facebook advertising) have been shown to have potential for aiding this effort [25]. Snowball effects would likely aid an ongoing recruitment process, as evidenced by a high percentage of pilot testers who indicated they would likely recommend the platform to others (82.2 and 77.9%).

ID Verification

Given the privacy issues attending healthcare information, providing security and assurances of confidentiality are of utmost importance to an online system. Prior to testing, we viewed the multiple layers of security involved in the process as both an advantage of the design (greater indications of data security) and a potential source of user dissatisfaction, as multiple steps can encumber the ease of use and accessibility of the process for all.

The lower rate of assent among the student group (59.8 vs. 82%) to the question of whether they would respond to the ID verification questions, along with comments shared in the simulation and exit surveys, indicate that the ID verification process had a negative impact on user experience. Helen Nissenbaum's notion of privacy as ‘contextual integrity' highlights the centrality of the norm of appropriateness that ‘dictate[s] what information about persons is appropriate, or fitting, to reveal in a particular context' [[41], p. 120]. Violations of this norm emerge when information from one situation is inserted into a different context [[41],p.122]. Feedback from our pilot testers suggests that, for some, the integration of personal financial data into the process of ID verification for health data constituted such a violation of the expected norms of appropriateness. Private Access has recently identified a new process for ID verification that uses the camera function on a computer or smart phone to capture an image of the user's state-issued identification (e.g. drivers license, passport). Several commercial vendors (e.g. Jumio, ID Checker) are already actively marketing this technology. Although Private Access has not yet implemented this approach, they believe that the familiarity that most people have with verifying their ID with government-issued ID will make users more comfortable than the process that was used in our pilot [Shelton, pers. commun.]

As technologies for ID verification continue to evolve and as public familiarity with the need for such tools, and the assurances they provide, increases, online ID verification, whether for e-commerce, banking or health IT applications, will become a more ubiquitous feature of our digital lives. The introduction of systems that rely on biometric authentication (e.g. facial recognition, fingerprint, etc.) rather than personal information gathered from a third party source, may minimize the kind of discomfort that some of our pilot testers experienced with our system. Providing privacy assurances for end-users will still have to be balanced with creating an atmosphere of trust. Determining appropriate security and validation measures that are scaled to the sensitivity of the information in question will be an important ongoing task for web-based PCIs.

Comparison of Group Outcomes

The variation in reactions and completion rates from the 2 participant groups suggests that a one-size-fits-all design approach may not be optimal for a consent portal. Given the complexity of the subject matter and the broad general population target audience, a design approach that enables users to toggle between different levels of information depth and relative ease of use may be a way to satisfy users across the full spectrum of literacy, tech literacy, interest and motivation to utilize a dynamic consent portal.

The ‘at-home' testing conditions are likely more indicative of how such a portal might fare if it were launched for the general public; thus, the reactions from participants in Group 1 may deserve greater weight when considering future iterations of a consent portal. The higher number of these participants who did not complete the simulation indicates that attention to the clarity and user-friendliness of the portal will be important to ensure that it has maximum efficacy. We were not able to follow up with participants who started, but did not complete the process, so we cannot say for sure why they did not complete it. Yet, the reactions of the student group are also highly instructive for future developments, as this group is in the age demographic that is most impacted by this particular project (‘donors' to the BioTrust's legacy collection are currently between the ages of 4 and 30).

Retrospective Consent: Is Consent Documentation Necessary?

An ongoing debate in the field, contested in the literature and in policy-making circles, is whether consent is necessary for research involving de-identified biospecimens, and if so, is it possible to obtain in a meaningful sense. Proposed changes to the Common Rule {45 CFR part 46a}, which currently excludes de-identified tissue samples from the category of human subjects research [13], would make broad, short-form consent necessary for secondary research uses of biospecimens. This proposed shift is itself controversial, though it was developed in light of research demonstrating that participants in biobanks strongly prefer to give consent for the secondary use of their samples and information. Further, the view of risks to individuals posed by donating samples to biobanks, and the need for enhancing protections, may be shifting given recent studies showing how research participants can be identified from ‘anonymous' DNA [42].

The goal of this project was to test the feasibility of creating an online portal for registering individuals' consent preferences regarding the disposition of their (or their children's) DBSs, with a particular focus on retrospective consent. With over 4 million residual DBS cards grandfathered into Michigan's biobank, the task of contacting, educating and consenting stakeholders with samples in this collection is challenging. Indeed, the waiver of consent granted to the Michigan Department of Community Health for secondary research use of de-identified DBSs was granted in consideration of the impracticability of such an effort [13].

At the same time that the waiver was granted for legacy DBSs, a consent process for prospectively collected DBSs was put in place, indicating recognition that, when possible, consented, educated donors are preferable to unconsented, uneducated donors when it comes to residual DBS research. The proposed change to the Common Rule appears to concede this point as well, although the merits of broad consent for biobanking are highly debatable [5,9,10,11,18,43]. In spite of the logistical burdens of contacting and consenting such a large population, one of the underlying rationales for conducting this project was that when it comes to donors to the legacy collection and donors to the prospective collection, the moral obligation is the same. Therefore, if a method or technology for overcoming the logistical burdens that separate the 2 groups could be tried, then it was minimally worth the effort, if not an ethical imperative.

Participant-Centric Initiatives

The impetus for developing a web-based consent portal was the recognition that the burden of recontacting, educating and consenting these individuals could be significantly diminished if the process were online. The longer term vision was that such a portal could be integrated into the biobank's data management systems so that each legacy DBS card, when digitized, could also be tagged with the individual's consent and recontact preferences. Experimentation with online, dynamic consent systems is ongoing [20,21,23,44] with notable recent projects recruiting individuals to submit genotypic and phenotypic information to web-based research biobanks with a very broad consent to perform research using these data (23andMe, Consent to Research - weconsent.us, Reg4All.org). More work will be required to determine if lessons learned from dynamic consent for prospective biobanks can apply equally to retrospective collections such as the BioTrust.

There are a number of potential benefits that an online consent portal could reap for a large population public biobank such as: augmenting existing outreach and consent processes with one that is accessible to all web-connected citizens, at all hours; automating the process of curating consent preferences for the biobank manager; offering a greater measure of personal control than is typically available to participants in biobanks; enabling a multimedia education component that may improve comprehension compared to traditional paper consent [45]; facilitating ongoing contact and communication between citizens and policy makers through integrating social networking platforms into the online system and enhancing the atmosphere of transparency and ultimately building trust.

The challenges moving forward for implementing a consent PCI for biobanking will include recruitment (i.e. developing a workable strategy to get individuals to use the portal) and ID verification (striking a delicate balancing between providing assurance of ID and creating an atmosphere of unwelcome intrusion into personal details). An additional challenge for gathering online consent for large population biobanks is that the extent to which individuals would want to be ‘involved' in the curation of their DBSs is largely unknown. Although much of the research shows that people strongly prefer to be asked permission for the use of their DBSs in research, other studies suggest that there is something of a paradox at work, where people want to retain control/autonomy yet do not want to be bothered with an overly burdensome or time-consuming process [27]. Can a balance be struck that facilitates a strong sense of participant autonomy and yet minimizes participant effort?

Online consent portals are promising tools for augmenting the role of participants as stewards of their own data that may increasingly be utilized as an answer to today's problems surrounding the practicability of contacting research participants and realizing the promises of genomic data. For large population biobanks, they will not alone solve the problem of unconsented participants, yet they may be an underappreciated tool to help these research goldmines to realize their potential and maintain trust with the public.

Limitations

Several limitations of the present study are worth noting as they limit the generalizability of the results. First, the sample of pilot testers was a convenience sampling and is, thus, not representative of the general public. The student group, in particular, was heavily concentrated in a single age and education bracket and was less racially and ethnically stratified than those in Group 1. However, this group had a stronger presence of females and had the added benefit of addressing the subject to individuals who are highly likely to have a DBS sample in Michigan's BioTrust. The sample size for the community participant group was lower than ideal, diminishing the statistical power of the results.

We are unable to report a response rate for Group 2 given the recruitment approach (outlined above). On large college campuses, it is common for students to have many opportunities to enroll in short-duration surveys and studies. As such, we followed a very typical mechanism of posting flyers in high-traffic locales and opening computer labs for drop-in participation in our pilot testing. It is not possible to determine (or reasonably estimate) the number of students who might have seen these posters or noticed the drop-in opportunities.

The process was monolingual. Through consultation with our community partners, we were able to set a target enrollment for each community, and in the case of the mostly Spanish speaking community in Detroit, we determined that the absence of a Spanish language option for the web portal would make this prohibitive for participants from that community. We, therefore, note that future web-focused work would benefit greatly from development and testing in multilingual contexts.

Finally, the hypothetical nature of the simulation made it somewhat challenging to articulate to users what their participation meant. Early feedback indicated that we required a clear and unambiguous declaration on all pages that users were not setting actual consent preferences. A major step that we were not able to assess with this pilot study was attempting to integrate these individual preferences into the data architecture of the biobank, a necessary next step that will yield invaluable knowledge about the challenges and affordances of ‘going live' with online consent.

Conclusion

By testing an online education and consent portal, we found that this approach holds promise as a means for communicating with biobank participants and facilitating active participation. More than half the pilot testers stated they would be likely to set up an account, and even larger numbers indicated likelihood to recommend the tool to others. The time to completion, ∼15-20 min, seemed appropriate to most participants, and most participants were comfortable with the ease of navigation through the tool. Challenges included striking the proper balance between promoting a sense of privacy and security while not overburdening users with security steps or utilizing verification techniques that inadvertently put users ill at ease. Scaling such a tool to serve a large population of biobank participants would require significant efforts to make the target population largely aware of the availability of such a system and determination of how best to incorporate such a system into the data management systems of the biobank.

Acknowledgements

This work was funded by an ARRA challenge grant issued through the National Human Genome Research Institute (5RC1HG005439-02). Additional support for this study was provided by a grant from the Eunice Kennedy Shriver National Institute for Child Health and Human Development (1R01HD067264). The authors gratefully acknowledge the community partner organizations, without whom this work would not have been possible: Community Based Organization Partners (Flint), The Asian Center (Grand Rapids), Arab Community Center for Economic and Social Services (Dearborn), Friends of Parkside (Detroit), Alliance Health (Jackson), and student participants from UM Ann Arbor. We also would like to acknowledge the leadership and staff at Private Access who were wonderful collaborators throughout this unique endeavor. Finally, we would like to thank the anonymous reviewers who made helpful and insightful suggestions for improving the manuscript.


References

  1. Tarini BA, Goldenberg A, Singer D, Clark SJ, Butchart A, Davis MM: Not without my permission: parents' willingness to permit use of newborn screening samples for research. Public Health Genomics 2010;13:125-130.
  2. Tarini BA: Storage and use of residual newborn screening blood spots: a public policy emergency. Genet Med 2011;13:619-620.
  3. Carmichael M: Newborn screening: a spot of trouble. Nature 2011;475:156-158.
  4. Rothwell E, Anderson R, Botkin J: Policy issues and stakeholder concerns regarding the storage and use of residual newborn dried blood samples for research. Policy Polit Nurs Tract 2010;11:5-12.
  5. Caplan A: What no one knows cannot hurt you: the limits of informed consent in the emerging world of biobanking; in Solbakk JH, Holm S, Hofmann B (eds): The Ethics of Research Biobanking. London, Springer, 2009, pp 25-32.
    External Resources
  6. Hofmann B, Solbakk JH, Holm S: Consent to biobank research: one size fits all?; in Solbakk JH, Holm S, Hofmann B (eds): The Ethics of Research Biobanking. London, Springer, 2009, pp 3-23.
    External Resources
  7. Clayton EW: Informed consent and biobanks. J Law Med Ethics 2005;33:15-21.
  8. Campbell AV: The ethical challenges of genetic databases: safeguarding altruism and trust. Kings Law J 2007;18:227-245.
  9. Kaye J, Whitley EA, Kanellopoulou N, Creese S, Hughes KJ, Lund D: Rapid response to: broad consent is informed consent. BMJ 2011;343:d6900.
  10. Shickle D: The consent problem within DNA biobanks. Stud Hist Philos Biol Biomed Sci 2006;37:503- 519.
  11. Petrini C: ‘Broad' consent, exceptions to consent and the question of using biological samples for research purposes different from the initial collection purpose. Soc Sci Med 2010;70:217-220.
  12. Langbo C, Bach J, Kleyn M, Downes FP: From newborn screening to population health research: implementation of the Michigan BioTrust for Health. Public Health Rep 2013;128:377-384.
    External Resources
  13. Mongoven A, McGee H: IRB review and public health biobanking: a case study of the Michigan BioTrust for Health. IRB 2012;34:11-16.
    External Resources
  14. Couzin-Frankel J: Newborn blood collections. Science gold mine, ethical minefield. Science 2009;324:166-168.
  15. Terry SF, Shelton R, Biggers G, Baker D, Edwards K: The haystack is made of needles. Genet Test Mol Biomarkers 2013;17:175-177.
  16. Shelton RH: Electronic consent channels: preserving patient privacy without handcuffing researchers. Sci Transl Med 2011;3:69cm4.
  17. Kaye J, Curren L, Anderson N, Edwards K, Fullerton SM, Kanellopoulou N, Lund D, MacArthur DG, Mascalzoni D, Shepherd J, Taylor PL, Terry SF, Winter SF: From patients to partners: participant-centric initiatives in biomedical research. Nat Rev Genet 2012;13:371-376.
  18. Steinsbekk KS, Kåre Myskja B, Solberg B: Broad consent versus dynamic consent in biobank research: is passive participation an ethical problem? Eur J Hum Genet 2013;21:897-902.
  19. Maschke KJ: Alternative consent approaches for biobank research. Lancet Oncol 2006;7:193-194.
  20. Kuehn BM: Groups experiment with digital tools for patient consent. JAMA 2013;310:678-680.
  21. Dixon WG, Spencer K, Williams H, Sanders C, Lund D, Whitley EA, Kaye J: A dynamic model of patient consent to sharing of medical record data. BMJ 2014;348:g1294.
  22. Stein DT, Terry SF: Reforming biobank consent policy: a necessary move away from broad consent toward dynamic consent. Genet Test Mol Biomarkers 2013;17:855-856.
  23. Wee R: Dynamic consent in the digital age of biology. J Prim Health Care 2013;5:259-261.
    External Resources
  24. O'Doherty KC, Burgess MM, Edwards K, Gallagher RP, Hawkins AK, Kaye J, McCaffrey V, Winickoff DE: From consent to institutions: designing adaptive governance for genomic biobanks. Soc Sci Med 2011;73:367-374.
  25. Platt J, Platt T, Thiel D, Kardia SLR: ‘Born in Michigan? You're in the Biobank': engaging population biobank participants through Facebook advertisements. Public Health Genomics 2013;16:145-158.
  26. Eder J, Gottweis H, Zatloukal K: IT solutions for privacy protection in biobanking. Public Health Genomics 2012;15:254-262.
  27. Simon CM, L'Heureux J, Murray JC, Winokur P, Weiner G, Newbury E, Shinkunas L, Zimmerman B: Active choice but not too active: public perspectives on biobank consent models. Genet Med 2011;13:821-831.
  28. Fleck LM, Mongoven A, Marzec S: Stored Blood Spots: Ethical and Policy Challenges. East Lansing, Michigan State University, 2008. http://ippsr.msu.edu/publications/ARBloodSpots.pdf (accessed March 17, 2014).
  29. Botkin JR, Goldenberg A, Rothwell E, Anderson R, Lewis M: Retention and research use of residual newborn screening bloodspots. Pediatrics 2013;131:120-127.
  30. Duquette D, Rafferty AP, Fussman C, Gehring J, Meyer S, Bach J: Public support for the use of newborn screening dried blood spots in health research. Public Health Genomics 2011;14:143-152.
  31. Duquette D, Langbo C, Bach J, Kleyn M: Michigan BioTrust for Health: public support for using residual dried blood spot samples for health research. Public Health Genomics 2012;15:146-155.
  32. Botkin JR, Rothwell E, Anderson R, Stark L, Goldenberg A, Lewis M, Burbank M, Wong B: Public attitudes regarding the use of residual newborn screening specimens for research. Pediatrics 2012;129:231-238.
  33. Rothwell E, Anderson R, Goldenberg A, Lewis M, Stark L, Burbank M, Wong B, Botkin JR: Assessing public attitudes on the retention and use of residual newborn screening blood samples: a focus group study. Soc Sci Med 2012;74:1305-1309.
  34. Murphy J, Scott J, Kaufman D, Geller G, LeRoy L, Hudson K: Public perspectives on informed consent for biobanking. Am J Public Health 2009;99:2128-2134.
  35. Institute for Public Policy and Social Research: State of the State Survey-60. 2011. http://www.ippsr.msu.edu/SOSS (accessed March 17, 2014).
  36. HHS.gov: Regulatory changes in ANPRM. 2013. http://www.hhs.gov/ohrp/humansubjects/anprmchangetable.html (accessed February 24, 2014).
  37. Thiel DB, Platt T, Platt J, King SB, Kardia SLR: Community perspectives on public health biobanking: an analysis of community meetings on the Michigan BioTrust for health. J Community Genet 2014;5:125-138.
  38. Boenigk S, Leipnitz S, Scherhag C: Altruistic values, satisfaction and loyalty among first-time blood donors. Int J Nonprofit Volunt Sect Mark 2011;16:356-370.
    External Resources
  39. Kaufman D, Murphy J, Erby L, Hudson K, Scott J: Veterans' attitudes regarding a database for genomic research. Genet Med 2009;11:329-337.
  40. Kaufman D, Murphy J, Scott J, Hudson K: Subjects matter: a survey of public opinions about a large genetic cohort study. Genet Med 2008;10:831-839.
  41. Nissenbaum H: Privacy as contextual integrity. Wash Law Rev 2004;79:119-158. http://ssrn.com/abstract=534622 (accessed March 17, 2014).
  42. Gymrek M, McGuire AL, Golan D, Halperin E, Erlich Y: Identifying personal genomes by surname inference. Science 2013;339:321-324.
  43. Wendler D: One-time general consent for research on biological samples. BMJ 2006;332:544-547.
  44. Kaye J, Whitley EA, Lund D, Morrison M, Teare H, Melham K: Dynamic consent: a patient interface for twenty-first century research networks. Eur J Hum Genet 2014, E-pub ahead of print.
  45. Henry J, Palmer B, Palinkas L, Glorioso DK, Caligiuri MP, Jeste DV: Reformed consent: adapting to new media and research participant preferences. IRB 2009;31:1-8.
    External Resources

Author Contacts

Daniel B. Thiel

Life Sciences and Society Program, University of Michigan School of Public Health

1415 Washington Heights, Suite 4605

Ann Arbor, MI 48109-2029 (USA)

E-Mail dbthiel@umich.edu


Article / Publication Details

First-Page Preview
Abstract of Original Paper

Received: March 28, 2014
Accepted: July 25, 2014
Published online: October 30, 2014
Issue release date: January 2015

Number of Print Pages: 14
Number of Figures: 2
Number of Tables: 4

ISSN: 1662-4246 (Print)
eISSN: 1662-8063 (Online)

For additional information: https://www.karger.com/PHG


Copyright / Drug Dosage / Disclaimer

Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.

References

  1. Tarini BA, Goldenberg A, Singer D, Clark SJ, Butchart A, Davis MM: Not without my permission: parents' willingness to permit use of newborn screening samples for research. Public Health Genomics 2010;13:125-130.
  2. Tarini BA: Storage and use of residual newborn screening blood spots: a public policy emergency. Genet Med 2011;13:619-620.
  3. Carmichael M: Newborn screening: a spot of trouble. Nature 2011;475:156-158.
  4. Rothwell E, Anderson R, Botkin J: Policy issues and stakeholder concerns regarding the storage and use of residual newborn dried blood samples for research. Policy Polit Nurs Tract 2010;11:5-12.
  5. Caplan A: What no one knows cannot hurt you: the limits of informed consent in the emerging world of biobanking; in Solbakk JH, Holm S, Hofmann B (eds): The Ethics of Research Biobanking. London, Springer, 2009, pp 25-32.
    External Resources
  6. Hofmann B, Solbakk JH, Holm S: Consent to biobank research: one size fits all?; in Solbakk JH, Holm S, Hofmann B (eds): The Ethics of Research Biobanking. London, Springer, 2009, pp 3-23.
    External Resources
  7. Clayton EW: Informed consent and biobanks. J Law Med Ethics 2005;33:15-21.
  8. Campbell AV: The ethical challenges of genetic databases: safeguarding altruism and trust. Kings Law J 2007;18:227-245.
  9. Kaye J, Whitley EA, Kanellopoulou N, Creese S, Hughes KJ, Lund D: Rapid response to: broad consent is informed consent. BMJ 2011;343:d6900.
  10. Shickle D: The consent problem within DNA biobanks. Stud Hist Philos Biol Biomed Sci 2006;37:503- 519.
  11. Petrini C: ‘Broad' consent, exceptions to consent and the question of using biological samples for research purposes different from the initial collection purpose. Soc Sci Med 2010;70:217-220.
  12. Langbo C, Bach J, Kleyn M, Downes FP: From newborn screening to population health research: implementation of the Michigan BioTrust for Health. Public Health Rep 2013;128:377-384.
    External Resources
  13. Mongoven A, McGee H: IRB review and public health biobanking: a case study of the Michigan BioTrust for Health. IRB 2012;34:11-16.
    External Resources
  14. Couzin-Frankel J: Newborn blood collections. Science gold mine, ethical minefield. Science 2009;324:166-168.
  15. Terry SF, Shelton R, Biggers G, Baker D, Edwards K: The haystack is made of needles. Genet Test Mol Biomarkers 2013;17:175-177.
  16. Shelton RH: Electronic consent channels: preserving patient privacy without handcuffing researchers. Sci Transl Med 2011;3:69cm4.
  17. Kaye J, Curren L, Anderson N, Edwards K, Fullerton SM, Kanellopoulou N, Lund D, MacArthur DG, Mascalzoni D, Shepherd J, Taylor PL, Terry SF, Winter SF: From patients to partners: participant-centric initiatives in biomedical research. Nat Rev Genet 2012;13:371-376.
  18. Steinsbekk KS, Kåre Myskja B, Solberg B: Broad consent versus dynamic consent in biobank research: is passive participation an ethical problem? Eur J Hum Genet 2013;21:897-902.
  19. Maschke KJ: Alternative consent approaches for biobank research. Lancet Oncol 2006;7:193-194.
  20. Kuehn BM: Groups experiment with digital tools for patient consent. JAMA 2013;310:678-680.
  21. Dixon WG, Spencer K, Williams H, Sanders C, Lund D, Whitley EA, Kaye J: A dynamic model of patient consent to sharing of medical record data. BMJ 2014;348:g1294.
  22. Stein DT, Terry SF: Reforming biobank consent policy: a necessary move away from broad consent toward dynamic consent. Genet Test Mol Biomarkers 2013;17:855-856.
  23. Wee R: Dynamic consent in the digital age of biology. J Prim Health Care 2013;5:259-261.
    External Resources
  24. O'Doherty KC, Burgess MM, Edwards K, Gallagher RP, Hawkins AK, Kaye J, McCaffrey V, Winickoff DE: From consent to institutions: designing adaptive governance for genomic biobanks. Soc Sci Med 2011;73:367-374.
  25. Platt J, Platt T, Thiel D, Kardia SLR: ‘Born in Michigan? You're in the Biobank': engaging population biobank participants through Facebook advertisements. Public Health Genomics 2013;16:145-158.
  26. Eder J, Gottweis H, Zatloukal K: IT solutions for privacy protection in biobanking. Public Health Genomics 2012;15:254-262.
  27. Simon CM, L'Heureux J, Murray JC, Winokur P, Weiner G, Newbury E, Shinkunas L, Zimmerman B: Active choice but not too active: public perspectives on biobank consent models. Genet Med 2011;13:821-831.
  28. Fleck LM, Mongoven A, Marzec S: Stored Blood Spots: Ethical and Policy Challenges. East Lansing, Michigan State University, 2008. http://ippsr.msu.edu/publications/ARBloodSpots.pdf (accessed March 17, 2014).
  29. Botkin JR, Goldenberg A, Rothwell E, Anderson R, Lewis M: Retention and research use of residual newborn screening bloodspots. Pediatrics 2013;131:120-127.
  30. Duquette D, Rafferty AP, Fussman C, Gehring J, Meyer S, Bach J: Public support for the use of newborn screening dried blood spots in health research. Public Health Genomics 2011;14:143-152.
  31. Duquette D, Langbo C, Bach J, Kleyn M: Michigan BioTrust for Health: public support for using residual dried blood spot samples for health research. Public Health Genomics 2012;15:146-155.
  32. Botkin JR, Rothwell E, Anderson R, Stark L, Goldenberg A, Lewis M, Burbank M, Wong B: Public attitudes regarding the use of residual newborn screening specimens for research. Pediatrics 2012;129:231-238.
  33. Rothwell E, Anderson R, Goldenberg A, Lewis M, Stark L, Burbank M, Wong B, Botkin JR: Assessing public attitudes on the retention and use of residual newborn screening blood samples: a focus group study. Soc Sci Med 2012;74:1305-1309.
  34. Murphy J, Scott J, Kaufman D, Geller G, LeRoy L, Hudson K: Public perspectives on informed consent for biobanking. Am J Public Health 2009;99:2128-2134.
  35. Institute for Public Policy and Social Research: State of the State Survey-60. 2011. http://www.ippsr.msu.edu/SOSS (accessed March 17, 2014).
  36. HHS.gov: Regulatory changes in ANPRM. 2013. http://www.hhs.gov/ohrp/humansubjects/anprmchangetable.html (accessed February 24, 2014).
  37. Thiel DB, Platt T, Platt J, King SB, Kardia SLR: Community perspectives on public health biobanking: an analysis of community meetings on the Michigan BioTrust for health. J Community Genet 2014;5:125-138.
  38. Boenigk S, Leipnitz S, Scherhag C: Altruistic values, satisfaction and loyalty among first-time blood donors. Int J Nonprofit Volunt Sect Mark 2011;16:356-370.
    External Resources
  39. Kaufman D, Murphy J, Erby L, Hudson K, Scott J: Veterans' attitudes regarding a database for genomic research. Genet Med 2009;11:329-337.
  40. Kaufman D, Murphy J, Scott J, Hudson K: Subjects matter: a survey of public opinions about a large genetic cohort study. Genet Med 2008;10:831-839.
  41. Nissenbaum H: Privacy as contextual integrity. Wash Law Rev 2004;79:119-158. http://ssrn.com/abstract=534622 (accessed March 17, 2014).
  42. Gymrek M, McGuire AL, Golan D, Halperin E, Erlich Y: Identifying personal genomes by surname inference. Science 2013;339:321-324.
  43. Wendler D: One-time general consent for research on biological samples. BMJ 2006;332:544-547.
  44. Kaye J, Whitley EA, Lund D, Morrison M, Teare H, Melham K: Dynamic consent: a patient interface for twenty-first century research networks. Eur J Hum Genet 2014, E-pub ahead of print.
  45. Henry J, Palmer B, Palinkas L, Glorioso DK, Caligiuri MP, Jeste DV: Reformed consent: adapting to new media and research participant preferences. IRB 2009;31:1-8.
    External Resources
ppt logo Download Images (.pptx)


Figures
Thumbnail
Thumbnail

Tables
Thumbnail
Thumbnail
Thumbnail
Thumbnail