Free Access
Cerebrovasc Dis 2012;34:424–429
(DOI:10.1159/000345077)

Comprehensive Stroke Centers and the ‘Weekend Effect’: The SPOTRIAS Experience

Albright K.C.a–d · Savitz S.I.e · Raman R.f · Martin-Schild S.h · Broderick J.i · Ernstrom K.f · Ford A.j · Khatri R.i · Kleindorfer D.i · Liebeskind D.k · Marshall R.l · Merino J.G.m · Meyer D.M.g · Rost N.n · Meyer B.C.g · on behalf of the SPOTRIAS Investigators
aHealth Services and Center for Outcome and Effectiveness Research and Education, bCenter of Excellence in Comparative Effectiveness Research for Eliminating Disparities, Minority Health and Health Disparities Research Center, cDepartment of Epidemiology, School of Public Health and dDepartment of Neurology, School of Medicine, University of Alabama at Birmingham, Birmingham, Ala., eDepartment of Neurology, University of Texas at Houston, Houston, Tex., fDivision of Biostatistics and Bioinformatics, Department of Family and Preventive Medicine and gDepartment of Neurosciences, UCSD, San Diego, Calif., hDepartment of Neurology, Tulane University School of Medicine, New Orleans, La., iDepartment of Neurology, University of Cincinnati, Cincinnati, Ohio, jCerebrovascular Disease Section, Department of Neurology, Washington University School of Medicine, St. Louis, Mo., kUCLA Stroke Center, Los Angeles, Calif., lColumbia University Medical Center, New York, N.Y., mSection on Stroke Diagnosis and Therapeutics, National Institute of Neurological Disorders and Stroke, NIH, Bethesda, Md., and nStroke Service, Department of Neurology, Massachusetts General Hospital, Boston, Mass., USA
email Corresponding Author


 Outline


 goto top of outline Key Words

  • The ‘weekend effect’
  • Acute stroke
  • Organized stroke care

 goto top of outline Abstract

Background and Purpose: Previous studies have found mortality among ischemic stroke patients to be higher on weekends. We sought to evaluate whether weekend admission was associated with worse outcomes in a large comprehensive stroke center (CSC) cohort. Methods: Consecutive ischemic stroke patients presenting within 6 h of symptom onset were identified using the 8 CSC SPOTRIAS (Specialized Programs of Translational Research in Acute Stroke) database. Patients who received intra-arterial therapy or who were enrolled in a nonobservational clinical trial were excluded. All patients meeting the inclusion criteria were then divided into two groups: weekday admissions or weekend admissions. Weekend admission was defined as Friday 17:01 to Monday 08:59. The remainder were classified as weekday admissions. Multivariate logistic regression was used, adjusting for age, stroke severity on admission [according to the National Institutes of Health Stroke Scale (NIHSS)] and admission glucose, in order to compare the outcomes of the weekend versus the weekday groups. Results: Eight thousand five hundred and eighty-one subjects from the combined SPOTRIAS database were screened from 2002 to 2009; 2,090 (24.4%) of these met the inclusion criteria. There was no significant difference in tissue plasminogen activator treatment rates between the weekday and weekend groups (58.5 vs. 60.4%, p = 0.397). Weekend admission was not a significant independent predictor of inhospital mortality (8.4 vs. 9.9%, p = 0.056), length of stay (4 vs. 5 days, p = 0.442), favorable discharge disposition (38.0 vs. 42.2%, p = 0.122), favorable functional outcome at discharge (41.6 vs. 43.4%, p = 0.805), favorable 90-day functional outcome (54.2 vs. 46.9%, p = 0.301), or 90-day mortality (18.2 vs. 19.8%, p = 0.680) when adjusting for age, NIHSS and admission glucose. Conclusions: In this large cohort of ischemic stroke patients treated at CSCs, we did not observe the ‘weekend effect.’ This may be due to access to stroke specialists 24 h a day on 365 days a year, nurses with stroke experience and the organized system for delivering care that is available at CSCs. These results suggest that EMS protocol should be reexamined regarding the preferential delivery of weekend stroke victims to hospitals that provide all levels of reperfusion therapy. This further highlights the importance of organized stroke care.

Copyright © 2012 S. Karger AG, Basel


goto top of outline Introduction

Stroke requires emergency medical care that is continuous, coordinated, timely and effective. In 2005, The Brain Attack Coalition (BAC) made recommendations for the designation of comprehensive stroke centers (CSC) as facilities with the necessary infrastructure, personnel, expertise and programs to diagnose and treat stroke patients who require a high intensity of medical and surgical care, specialized tests or interventional therapies [1]. While the BAC suggested that CSCs may parallel trauma centers in their ability to enhance treatment and improve patient outcome, little evidence with regard to patient outcome exists for CSCs.

Data from prior epidemiologic studies have raised concerns that care for stroke patients may be inconsistent and perhaps even inferior during off-hours compared to during regular business hours [2,3,4,5,6,7,8,9,10,11,12,13,14]. This has since been dubbed the ‘weekend effect’. National data from Canada found a worse discharge disposition, an increased 7-day case fatality and increased inhospital mortality in ischemic stroke patients admitted on the weekend [8]. Similar results were reported in both a Swedish national sample and in the US Nationwide Inpatient Sample of Healthcare Cost and Utilization Project [7,13]. More recently, Reeves et al. [10 ] reported increased inhospital mortality for weekend admissions for acute ischemic stroke (AIS) patients using Get with the Guidelines data. It has been postulated that the ‘weekend effect’ observed in AIS, myocardial infarction and other serious medical conditions, may be related to reduced staffing levels, a decreased availability of resources and diminished access to subspecialty care on weekends [15,16]. Not all studies, however, have confirmed the ‘weekend effect’ and further study is needed to determine whether there truly is a reduced quality level after hours.

SPOTRIAS (Specialized Programs of Translational Research in Acute Stroke) is a national network of stroke centers developed and deployed with funding from the National institutes of Health/National Institute of Neurological Disorders and Stroke (NIH/NINDS) [17]. SPOTRIAS was designed to perform early-phase clinical projects, collect and share registry level data and promote new approaches to therapy for acute stroke. This network of academic tertiary care medical centers provides acute stroke therapies for thousands of patients, participates in cutting-edge research for acute stroke and provides the options for surgical and interventional management of patients above and beyond standard medical therapies. These centers have the resources of CSCs, and thus provide a unique opportunity to explore the ‘weekend effect’ in a CSC environment. In addition, most SPOTRIAS centers collect data from nearby community medical centers (‘spokes’) allowing a cross-representational study of a ‘weekend effect’ in a variety of medical settings.

Our previous work failed to find evidence of inferior outcomes associated with the ‘weekend effect’ in transient ischemic attack, AIS or intracerebral hemorrhage patients when two of the eight SPOTRIAS sites were examined [18]. Given the estimation by Reeves et al. [10 ] that 1/20 inhospital stroke deaths could be avoided if the system were to eliminate the higher mortality associated with off-hours presentation, the aim of this study was to assess patient outcomes in a multi-site collaboration of CSCs and to determine if patients presenting on weekends indeed have worse outcomes, or if evaluation by CSC stroke specialists could help to erase the ‘weekend effect’. We hypothesized that centers with continuous availability of expert stroke teams, necessary diagnostic and therapeutic modalities and stroke-nursing expertise would have consistent patient outcomes, regardless of the day of presentation.

 

goto top of outline Materials and Methods

SPOTRIAS is a network of stroke centers from 8 different sites (in alphabetical order: Columbia University, Intramural NINDS Program, Partners at Massachusetts General, University of California Los Angeles, University of California San Diego, University of Cincinnati, University of Texas Houston and Washington University) which share data elements in a common SPOTRIAS clinical and neuroimaging database. SPOTRIAS centers meet the CSC criteria as outlined by the BAC. While sites may differ in their practice patterns, each site provides access to stroke specialists and nurses with specialized training 24 h a day on 365 days of the year. In their role as CSCs, SPOTRIAS sites may serve as a hub for outlying spoke hospitals, e.g. primary stroke centers (PSCs) and nonstroke centers (NSCs).

From the common SPOTRIAS database, we identified ischemic stroke patients arriving within 6 h of symptom onset, in an effort to eliminate patients with excessive delay in presentation. Patients who had received intra-arterial therapy or who were enrolled in a nonobservational clinical trial were excluded, as these interventions have an undetermined impact on outcomes. All patients meeting the inclusion criteria were then divided into two groups: weekday and weekend admissions. Weekend admission was defined as the period from Friday at 17:01 to Monday at 08:59. All other admissions were classified as weekday admissions.

Patient age, gender, race/ethnicity, admission NIHSS (National Institutes of Health Stroke Scale) score and admission glucose levels were examined. Outcome measures included length of hospital stay (LOS), inhospital mortality, discharge disposition (home favorable vs. all others unfavorable) and functional outcome upon discharge and at 90-day follow-up, as measured by the modified Rankin Scale (mRS, 0–2 = favorable vs. 3–6 = poor), and 90-day mortality. We compared all the outcome measures of the weekend and weekday groups.

Demographic and baseline characteristics between weekend admissions and weekday admissions were compared using the Fisher’s exact test for categorical variables and the Wilcoxon rank sum test for continuous variables. Logistic regression or ANOVA, as appropriate, was used to test the interaction of site and weekend status. The binary outcome variables inhouse mortality, discharge mRS, 90-day mortality and 90-day mRS of weekend and weekday admissions were compared using the multivariable logistic regression, adjusting for admission NIHSS, age and admission glucose. Site was included as a covariate in the multivariable model if the Cochran-Mantel-Haenszel χ2 test stratified by site comparing the outcome rates between weekend and weekday admissions was statistically significant at the 0.05 level. The LOS of the two groups was compared using the Wilcoxon rank sum test.

As this was an exploratory analysis, no adjustments for multiple comparisons were made and a p value of <0.05 was considered to be statistically significant. To assess the differences between groups for those outcome variables that did not show a statistical significance, we chose to use 95% confidence intervals instead of a post hoc power calculation [19]. All statistical analysis was conducted using the statistical software R, version 2.6.2.

 

goto top of outline Results

A total of 8,581 consecutive patients were screened from 2002 to 2009. Of these, 2,090 from the combined SPOTRIAS database met the inclusion criteria (in random order: n = 278 at site 1, n = 99 at site 2, n = 872 at site 3, n = 268 at site 4, n = 150 at site 5, n = 348 at site 6, n = 43 at site 7 and n = 32 at site 8). Patient demographics in the weekend and weekday admission groups are shown in table 1. Overall, the mean age was 68.3 years (SD 15, n = 2,085). Over half (52.7% or 1,094/2,077) of the participants were male. There was no significant difference in the mean age or gender for patients admitted on weekends (68.2 vs. 68.3, p = 0.972) and weekdays, (51.0 vs. 53.6%, p = 0.268). The majority of patients were Caucasian/White (60.0% or 1,254/2,090), followed by African American/Black (22.2% or 463/2,090), Hispanic non-Black (9.4% or 196/2,090), other (5.7% or 119/2,090), and Asian (2.8% or 58/2,090) with no significant differences in the race/ethnicity of patients arriving on the weekend versus those arriving on weekdays (p = 0.944). The median NIHSS score was 8, ranging from 0 to 40 (IQR 4, 15; n = 1,892) and the mean admission glucose level was 141 mg/dl (SD 64, n = 1,943). There was no significant difference in the median NIHSS score for patients admitted on weekends and weekdays (9 vs. 8, p = 0.157) or the mean admission glucose levels for weekend versus weekday patients (143.5 vs. 139.8, p = 0.313). Overall, 59.1% of patients arriving within 6 h of symptom onset were treated with intravenous tissue plasminogen activator (IV t-PA). There was no significant difference in IV t-PA treatment rates in patients presenting on a weekend and those presenting on a weekday (60.4 vs. 58.5%, p = 0.397).

TAB01
Table 1. Patient demographics

Patient outcomes comparing the weekend and weekday admission groups are shown in table 2. As specified a priori, all outcome analyses were adjusted for patient age, admission NIHSS and admission glucose. Inhospital mortality overall was 9.3% (141/1,509). Patients admitted on weekends did not have a higher rate of inhospital mortality; in fact, there was a trend suggesting higher mortality amongst patients admitted on weekdays (8.4 vs. 9.9%, p = 0.056). The median LOS was 5 days, ranging from 0 to 111 days (IQR 3, 7). There was no significant difference in the median LOS in patients admitted on weekends and those admitted on weekdays (4 vs. 5 days, p = 0.442). Over 40% of patients were discharged and sent home (40.7%, 851/2,089) with no significant difference in favorable discharge disposition between the weekend and weekday groups (38.0 vs. 42.2%, p = 0.122). Nearly 43% of patients had a favorable functional outcome at discharge (42.7%, 645/1,509) with no evidence of significant differences between those admitted on weekends and those admitted on weekdays (41.6 vs. 43.4%, p = 0.805) after adjusting for age, admission NIHSS, admission glucose and site. Nearly half of the patients had a favorable 90-day functional outcome (49.3%, 369/748). No significant difference was found between weekend and weekday patients in the favorable functional outcome at 90 days when controlling for age, NIHSS on admission and glucose (54.2 vs. 46.9%, p = 0.301). Mortality at 90-day follow-up was 19.3% (144/748). Patients admitted on weekends had a similar 90-day mortality to their weekday counterparts (18.2 vs. 19.8%, p = 0.680).

TAB02
Table 2. Patient outcomes

 

goto top of outline Discussion

To the best of our knowledge, this is the largest study of CSCs that has examined the impact of the day of presentation on patient outcomes. Our study found no evidence of the ‘weekend effect’. Ischemic stroke patients admitted on weekends did not display significant differences in LOS, inhospital mortality, favorable hospital discharge or functional outcome at discharge or an inferior 90-day functional outcome or higher 90-day mortality.

Our results differ from previously published studies that reported an increased LOS [13], inferior functional outcomes [6], inferior discharge disposition [7,10] and increased mortality rates [7,8,10,11,12] in ischemic stroke patients presenting on the weekend or during off-hours. One possible explanation for this difference is the homogenous nature of our sample. While our study included only centers that met BAC criteria for CSCs, previous studies included a heterogeneous sample of hospitals (e.g. state-designated CSCs and PSCs, self-designated CSCs, The Joint Commission PSCs and NSCs).

Our findings concur with those of Kazley et al. [20] and Streifler et al. [21] who found no statistically significant difference in stroke mortality based on day of admission. As demonstrated in our previous work, analysis of this larger multicenter sample suggests that centers that focus on providing comprehensive care around the clock irrespective of the day of the week may be less susceptible to the ‘weekend effect’ [18].

We have shown that by providing access to stroke specialists, stroke teams and organized systems of care, we can reduce the ‘weekend effect’. Our findings have significant health policy implications. Using the calculation proposed by Reeves et al. [10], use of CSCs might prevent 1/20 inhospital stroke deaths by eliminating the higher mortality associated with off-hours presentation. It is possible that the previously postulated causes of the ‘weekend effect’, such as reduced staffing levels, a decreased availability of resources and diminished access to subspecialty care on weekends can be averted by the continuous availability of expert stroke teams and the necessary diagnostic and therapeutic modalities and stroke-nursing expertise afforded by CSCs.

Our study is not without limitations. While our data were collected prospectively, this study remains a retrospective analysis. As it was an exploratory analysis, we did not control for multiple comparisons. Low event rates may limit our ability to detect differences. In light of the International Stroke Trial, patients arriving within the 6-hour treatment window were selected. Our aim was to determine if the exposure of arriving on a weekend placed ischemic stroke patients at increased odds of having a poor outcome. To do this, we elected to limit our sample to patients arriving within 6 h of symptom onset. Thus, our results are not generalizable to other types of stroke (e.g. transient ischemic attack and intracerebral hemorrhage) or to patients arriving more than 6 h from symptom onset. Furthermore, since intra-arterial recanalization is yet to be fully proven to improve outcome or has yet to receive consensus on its benefits compared to IV t-PA, we also excluded this variable. Lastly, we did not include nonobservational trials as, by their very nature, these approaches have not yet been proven to affect stroke outcome. While excluding patients arriving beyond 6 h, those receiving intra-arterial therapy and those enrolled in acute-stroke therapy trials served to limit potential confounders of patient outcome, it may have resulted in our rather high weekend admission rate. While one cannot be certain, it is possible that a higher proportion of patients may have presented within 6 h on the weekend. Another possibility is that a higher number of patients were enrolled on weekdays, thus making fewer weekday patients eligible for our study. A third possibility is that more patients were taken for intra-arterial therapy on weekdays, which would also make them ineligible for our study.

Using SPOTRIAS data provides the benefits of a multi-site collaboration (e.g. a nationally representative sample and increased sample size); however, practice patterns may vary at the 8 individual institutions. While all the sites meet the BAC criteria for CSCs, SPOTRIAS stroke specialists may admit patients to a primary stroke service, a general neurology service or simply serve as consultants in the acute management of stroke patients. In addition, each site within the SPOTRIAS network may serve as a hub hospital to one or more spoke hospitals. Despite differences in patterns of practice and the hub and spoke sub-network established at our sites, the outcomes for weekend patients cared for by CSC stroke specialists appear to differ from previously reported outcomes of patients admitted to NSCs and PSCs on the weekend.

Our results suggest that patients evaluated by CSC specialists do not experience the ‘weekend effect’. Our findings raise the important question of whether EMS policy on hospital selection for acute stroke victims should be modified to permit diversion to CSCs and/or CSC spoke hospitals on weekends. It may be advisable for centers that cannot provide around-the-clock acute-stroke care to vigilantly seek help from centers that are able to do so, particularly on weekends and after hours, as this may help to prevent 1/20 unnecessary deaths [10] and potentially decrease long-term stroke morbidity.

 

goto top of outline Acknowledgements

The authors would like to thank Patrick D. Lyden, MD, for his invaluable assistance with study design, interpretation of data and critical revision of the manuscript and Sharyl Martini, MD, PhD for her support and assistance. The study was supported by P50 NS044227. The research was supported, in part, by the Intramural Research Program of the NINDS, NIH as well as by Award Nos. 5 T32 HS013852-10 from The Agency for Healthcare Research and Quality (AHRQ) and 3 P60 MD000502-08S1 from The National Institute on Minority Health and Health Disparities, NIH. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or the NIH.


 goto top of outline References
  1. Alberts MJ, Latchaw RE, Selman WR, Shephard T, Hadley MN, Brass LM, Koroshetz W, Marler JR, Booss J, Zorowitz RD, Croft JB, Magnis E, Mulligan D, Jagoda A, O’Connor R, Cawley CM, Connors JJ, Rose-DeRenzy JA, Emr M, Warren M, Walker MD: Recommendations for comprehensive stroke centers: a consensus statement from the Brain Attack Coalition. Stroke 2005;36:1597–1616.
  2. Chambers BR, Donnan GA, Bladin PF: Patterns of stroke. An analysis of the first 700 consecutive admissions to the Austin Hospital Stroke Unit. Aust N Z J Med 1983;13:57–64.
  3. Langhorne P Williams BO, Gilchrist W, Howie K: Do stroke units save lives? Lancet 1993;342:395–398.
  4. Collaboration. SUT: Organised inpatient (stroke unit) care for stroke. Cochrane Database Syst Rev 2000;2:CD000197.
  5. Schmidt WP, Taeger D, Buecker-Nott HJ, Berger K: The impact of the day of the week and month of admission on the length of hospital stay in stroke patients. Cerebrovasc Dis 2003;16:247–252.
  6. Hasegawa Y, Yoneda Y, Okuda S, Hamada R, Toyota A, Gotoh J, Watanabe M, Okada Y, Ikeda K, Ibayashi S: The effect of weekends and holidays on stroke outcome in acute stroke units. Cerebrovasc Dis 2005;20:325–331.
  7. Janszky I, Ahnve S, Ljung R: Weekend versus weekday admission and stroke outcome in Sweden from 1968 to 2005. Stroke 2007;38:e94.
  8. Saposnik G, Baibergenova A, Bayer N, Hachinski V: Weekends: a dangerous time for having a stroke? Stroke 2007;38:1211–1215.
  9. Turin TC, Kita Y, Rumana N, Ichikawa M, Sugihara H, Morita Y, Tomioka N, Okayama A, Nakamura Y, Ueshima H: Case fatality of stroke and day of the week: is the weekend effect an artifact? Cerebrovasc Dis 2008;26:606–611.
  10. Reeves MJ, Smith E, Fonarow G, Hernandez A, Pan W, Schwamm LH: Off-hour admission and in-hospital stroke case fatality in the Get with the Guidelines Stroke Program. Stroke 2009;40:569–576.
  11. Tung YC, Chang GM, Chen YH: Associations of physician volume and weekend admissions with ischemic stroke outcome in Taiwan: a nationwide population-based study. Med Care 2009;47:1018–1025.
  12. Fang J, Saposnik G, Silver FL, Kapral MK: Association between weekend hospital presentation and stroke fatality. Neurology 2010;75:1589–1596.
  13. Hoh BL, Chi Y-Y, Waters MF, Mocco J, Barker FG II: Effect of weekend compared with weekday stroke admission on thrombolytic use, in-hospital mortality, discharge disposition, hospital charges, and length of stay in the nationwide inpatient sample database, 2002 to 2007. Stroke 2010;41:2323–2328.
  14. Ogbu UC, Westert GP, Slobbe LC, Stronks K, Arah OA: A multifaceted look at time of admission and its impact on case-fatality among a cohort of ischaemic stroke patients. J Neurol Neurosurg Psychiatry 2011;82:8–13.
  15. Bell CM RD: Mortality among patients admitted to hospitals on weekends as compared with weekdays. N Engl J Med 2001;345:663–668.
  16. Kostis WJ DK, Marcella SW, Shao YH, Wilson AC, Moreyra AE, Myocardial Infarction Data Acquisition System (MIDAS 10) Study Group: Weekend versus weekday admission and mortality from myocardial infarction. N Engl J Med 2007;356:1099–1109.
  17. InternetStrokeCenter: Specialized Programs of Translational Research in Acute Stroke (SPOTRIAS).
  18. Albright KC, Raman R, Ernstrom K, Hallevi H, Martin-Schild S, Meyer BC, Meyer DM, Morales MM, Grotta JC, Lyden PD, Savitz SI: Can comprehensive stroke centers erase the ‘weekend effect’? Cerebrovasc Dis 2009;27:107–113.
  19. Hoenig J, Heisey D: The abuse of power: the pervasive fallacy of power calculations for data analysis. Am Stat 2001;55:19–24.

    External Resources

  20. Kazley AS, Hillman DG, Johnston KC, Simpson KN: Hospital care for patients experiencing weekend vs weekday stroke: a comparison of quality and aggressiveness of care. Arch Neurol 2010;67:39–44.
  21. Streifler JY, Benderly M, Molshatzki N, Bornstein N, Tanne D: Off-hours admission for acute stroke is not associated with worse outcome – a nationwide Israeli stroke project. Eur J Neurol 2012;19:643–647.

 goto top of outline Author Contacts

Karen C. Albright
RWUH M226
619 19th Street South
Birmingham, AL 35249-3280 (USA)
E-Mail kca@uab.edu


 goto top of outline Article Information

K.C.A. and S.I.S. contributed equally to this article as first authors.

Received: June 8, 2012
Accepted: October 12, 2012
Published online: December 1, 2012
Number of Print Pages : 6
Number of Figures : 0, Number of Tables : 2, Number of References : 21


 goto top of outline Publication Details

Cerebrovascular Diseases

Vol. 34, No. 5-6, Year 2012 (Cover Date: December 2012)

Journal Editor: Hennerici M.G. (Mannheim)
ISSN: 1015-9770 (Print), eISSN: 1421-9786 (Online)

For additional information: http://www.karger.com/CED


Copyright / Drug Dosage / Disclaimer

Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher or, in the case of photocopying, direct payment of a specified fee to the Copyright Clearance Center.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in goverment regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.