Research Methodology Reference Guide¶

The Turabian Citation System¶

Overview¶

Kate L. Turabian created the manual in 1937 to simplify the Chicago Manual of Style for students. The 9th edition (2018) aligns with Chicago's 17th edition and remains current as of June 2026. Two systems, one underlying logic.

Current edition note: CMOS 18th edition (2024) has already changed some rules (notably eliminating the 3-em dash in favor of repeating author names), but Turabian has not been updated past the 9th edition — the 3-em dash remains correct for Turabian as of June 2026. When Turabian updates, re-verify.

System 1: Notes-Bibliography (NB) Style¶

Used in: humanities, literature, history, arts (Turabian chapters 16–17).

Mechanism: superscript Arabic numeral in the text → footnote (bottom of page, preferred) or endnote (end of paper/chapter), plus a Bibliography at the back. Footnotes are generally preferred over endnotes because readers can follow them more easily.

Full Note vs. Bibliography Entry — The Critical Punctuation Differences¶

Element	Full Note	Bibliography Entry
Author name order	First Last (natural order)	Last, First (inverted — first author only)
Element separators	Commas	Periods
Publication data	Wrapped in parentheses	No parentheses
Cited page	Specific page cited	Full page range (for articles/chapters)
Indentation	Number + hanging indent	Hanging indent
Number	Present (superscript or numeral + period)	Absent

Examples (from official Turabian Quick Guide): - Full note: 1. Katie Kitamura, A Separation (New York: Riverhead Books, 2017), 25. - Bibliography: Kitamura, Katie. A Separation. New York: Riverhead Books, 2017.

Shortened Notes — The 9th Edition's Most Important Change¶

The 9th edition de-emphasizes "Ibid." in favor of shortened notes for all repeated citations (§16.4.2). Format: author's last name + short title (up to 4 distinctive words, italicized for books / in quotation marks for articles) + page.

Examples: - Book: 3. Kitamura, Separation, 91–92. - Two authors: 4. Sassler and Miller, Cohabitation Nation, 205. - Journal article: 4. Pérez, "Material Morality," 880–81.

"Ibid.": remains permitted for a note immediately following another to the same work, but the 9th edition discourages it — the official Quick Guide uses no Ibid. at all. Institutional override: some institutions (e.g., Liberty University) still mandate Ibid. for consecutive same-page citations — always check house style.

The 3-Em Dash for Repeated Authors¶

When listing multiple works by the same author in a bibliography, replace the name after the first entry with three em dashes followed by a period:

Smith, Mary. First Book. New York: Publisher, 2011.
———. Second Book. New York: Publisher, 2012.

(Type the em dash three times, or use six hyphens.) Applies to Turabian 9th / CMOS 17 — but NOT to CMOS 18.

System 2: Author-Date Style¶

Used in: physical, natural, and social sciences (Turabian chapters 18–19).

Mechanism: parenthetical in-text citation (author last name, year, page) → "References" list at the back. - Example: (Johns 1998, 623)

Everything else about full reference construction follows the same logic as NB, but with author-date substituting for footnote numbering.

Major Source Type Formats (Notes-Bibliography)¶

Book (single author) - Note: 1. Firstname Lastname, Title of Book (Place: Publisher, Year), page. - Bibliography: Lastname, Firstname. Title of Book. Place: Publisher, Year.

Journal Article - Note: 1. Firstname Lastname, "Article Title," Journal Name volume, no. issue (Month/Season Year): page, DOI. - Use the DOI over the address-bar URL when available (forms a permanent URL: https://doi.org/...). - For continuously-paginated journals, the issue number may be omitted.

Multiple Authors - Two or three: list all; invert only the first author in the bibliography; join with "and" (not "&"). - Four or more: note gives first author + "et al."; bibliography lists up to ten (first seven + "et al." if more than ten).

Chapter in Edited Book - Note cites specific pages; bibliography gives chapter's full page range. - Use "edited by" before editor's name in bibliography.

Website - Author, "Page Title," Publishing Organization/Website Name, publication or modified date, URL. - Include an access date only for content likely to change (wikis, etc.).

Dissertation/Thesis - Title in quotation marks (not italics); degree + institution + year in parentheses. - Example: 1. Guadalupe Navarro-Garcia, "Integrating Social Justice Values in Educational Leadership..." (PhD diss., University of California, Los Angeles, 2016), 44, ProQuest Dissertations & Theses Global.

Interviews and Personal Communications - Personal interviews, emails, texts, DMs: usually cited only in the text or a note, not the bibliography. - Example note: 1. Sam Gomez, Facebook message to author, August 1, 2017. - Published/archived oral-history interviews: get full citations including URL.

Archival Material - No single correct format; the priority is to describe the item to uniqueness. - General pattern: Author/creator, title or description, date, collection name, box/folder, repository, location, URL if online. - Example: Caroline Howard Gilman to Anna M. White, 15 January 1859, Caroline Howard Gilman Papers (1036.00), South Carolina Historical Society, Charleston, SC.

Government Documents - Follow Turabian's public-documents models; include issuing body, title, and publication data.

Scripture and Classical Works - Cited in text or notes only (not the bibliography), using standard abbreviations in parentheses — e.g., (Rom. 1:16).

Formatting Conventions¶

Body: double-spaced.
Footnotes, block quotes, bibliography entries: single-spaced with a blank line between entries.
Block quotes: for quotations longer than ~5 lines; indent without quotation marks.
Title capitalization: headline-style (capitalize major words).
Tense: past tense for historical events; present tense when describing what an author argues.
DOI vs. URL: always prefer DOI for journal articles.

Research Design¶

Choosing a Design¶

Type	Purpose	Appropriate When
Quantitative	Test theories, measure variables numerically	Hypotheses are clear, constructs are measurable, generalization is needed
Qualitative	Explore meaning, process, experience	Questions are "how" or "why," phenomena are poorly understood
Mixed-methods	Both	Neither alone answers the full question; triangulation adds value

Mixed-Methods Designs¶

Convergent (parallel): collect both simultaneously, analyze separately, then compare/merge. Use when time is limited and you want to validate one against the other.
Explanatory sequential: quantitative first, then qualitative to explain the numbers in depth.
Exploratory sequential: qualitative first to surface themes, then quantitative to test them at scale — ideal for new product/instrument development.

Triangulation: convergence of findings across methods increases confidence; divergence is itself informative.

Constructing Research Questions¶

PICO Framework¶

Element	Definition
P	Population — who are the participants?
I	Intervention — what is being done?
C	Comparison — what is the control or alternative?
O	Outcome — what is being measured?

Structures a focused, answerable question for comparative/clinical studies. Distinguishes "background" questions (answered by textbooks/reviews) from "foreground" questions (answered by primary studies).

FINER Framework¶

Evaluates whether a question is worth pursuing: - Feasible: achievable with available resources and subjects? - Interesting: will it engage the field? - Novel: does it add to, confirm, or refute existing knowledge? - Ethical: can it be done with appropriate protections? - Relevant: does it matter to the field and/or practice?

Invest heavily in question framing — a poorly framed question creates downstream problems that surface only at peer review.

Hypotheses¶

Derive from the research question: a null hypothesis (no difference/relationship) and an alternative hypothesis (directional or non-directional). Preregistering both before data collection is best practice.

Literature Reviews: Systematic vs. Narrative¶

These are different study types, not better/worse versions of each other.

Feature	Systematic Review	Narrative Review
Scope	Narrow, predefined question	Broader, traces concept development
Protocol	PRISMA, Cochrane Handbook	No mandated guidelines
Searching	Comprehensive, multi-database, includes grey literature	Author judgment
Reproducibility	Documented search log + PRISMA flow diagram	Often not reproducible
Use case	Best-practice recommendations, policy	Topic too broad, literature sparse, invited expert review

PRISMA: preferred reporting standard for systematic reviews — includes a checklist and flow diagram (identification → screening → eligibility → included).

Sampling¶

Probability Sampling (enables generalization)¶

Simple random: every member has an equal chance.
Systematic: every nth member.
Stratified: ensures proportional representation of subgroups; proportional or disproportional.
Cluster: sample groups (clusters), then all or random members within.
Multistage: combines cluster and simple random.

Non-Probability Sampling (researcher judgment/accessibility)¶

Convenience: easiest to access; carries selection-bias risk; common in exploratory/qualitative work.
Quota: ensures representation of categories but not probability-based.
Purposive/judgmental: deliberately selecting information-rich cases.
Snowball: referral chains for hidden/hard-to-reach populations.
Voluntary response: self-selection bias risk.

Most common avoidable errors: defaulting to convenience sampling without justification; overgeneralizing from non-probability samples; failing to justify sample size.

Validity and Reliability¶

The Core Distinction¶

Reliability = consistency (a measure repeated yields the same result).
Validity = accuracy (you measure what you claim to measure).
You can be reliable but invalid (a scale consistently 5 lbs off).

Types of Validity¶

Type	Question
Internal validity	Can you attribute cause to effect, ruling out confounds? Randomization is the strongest tool.
External validity	Can findings generalize to other people (population validity) and settings (ecological validity)?
Construct validity	Does your instrument measure the intended abstract concept?
Criterion validity	Does it correlate with an external gold standard?
Content validity	Does it cover the full domain of the construct?
Face validity	Does it appear to measure what it claims? (Weakest form)

The fundamental trade-off: tightly controlled lab settings raise internal validity but lower external validity.

Threats to Internal Validity¶

History, maturation, instrumentation changes, testing effects, selection bias, regression to the mean, and attrition/mortality.

Ethics and IRB¶

The Belmont Report (1979)¶

Codified in U.S. regulation as the Common Rule (45 CFR 46). Three principles:

Respect for persons → voluntary participation + informed consent (three elements: information, comprehension, voluntariness).
Beneficence → do no harm; maximize benefits, minimize harms.
Justice → fair distribution of research burdens and benefits; do not exploit vulnerable populations.

IRB Review¶

An Institutional Review Board must review human-subjects research before it begins.
Minimal-risk studies may qualify for expedited review.
Written consent may be waived when the signed form is the only identifying link, or for minimal-risk survey/interview research.
2018 Revised Common Rule: added requirements including informing subjects about possible future commercial profit from their data.

Rule: obtain IRB approval before any data collection, including screening.

Academic Writing Standards¶

IMRaD Structure¶

The dominant structure for empirical papers (Introduction, Methods, Results, and Discussion). Follows an hourglass shape: broad context → narrow to your specific question → methods/results at the narrowest → broaden again in discussion.

Section	Contents
Introduction	Problem → current state of the field → the gap → how your study fills it; ends with research objective/hypothesis
Methods	Enough detail for replication; past tense; justify design, sample, instruments, analysis
Results	Report findings only — no interpretation; past tense
Discussion	Interpret, compare to literature, state implications and limitations, suggest future work

Common failures: abstract lacking the main finding; introduction that doesn't justify importance; disorganized methods; results section that editorializes.

Medawar critique (1964 BBC talk "Is the Scientific Paper a Fraud?"): the orthodox IMRaD form misrepresents the actual process of scientific thought — "a totally mistaken conception, even a travesty, of the nature of scientific thought." IMRaD is a reporting convention, not a research diary.

Tone and Hedging¶

Academic prose: objective and precise. Calibrated hedging ("these data suggest," "may indicate," "is consistent with") is a feature, not weakness. Over-claiming is a common reviewer target.

Turabian recommends third person and active voice unless the instructor specifies otherwise.

Avoiding Plagiarism: The Patchwriting Trap¶

Patchwriting: changing a few words or reordering phrases while keeping the source's structure or vocabulary. This is plagiarism even when cited because it misrepresents whose words the reader is seeing.

The Four-Step Paraphrase Method¶

Read the source until you understand it.
Put it away and write from memory.
Compare and rewrite anything still too close — phrases longer than ~4 words should be reworded or quoted.
Verify you haven't misrepresented the original.

When to Quote vs. Paraphrase¶

Quote: when exact wording is essential or distinctive — always use quotation marks + citation; use sparingly.
Paraphrase: when you can restate the idea in a genuinely new structure — citation required, no quotation marks.
Synthesis — combining multiple sources into your own argument — is the higher-order skill; substituting synonyms is not synthesis.

Source Evaluation¶

CRAAP Test (Blakeslee, CSU Chico)¶

Currency: timeliness of the information.
Relevance: fit to your specific question.
Authority: who created it; credentials; institutional affiliation.
Accuracy: evidence, references, verifiability, peer review.
Purpose: to inform vs. persuade vs. sell; potential bias.

Treat as a guide, not a rigid checklist — you can legitimately use sources that "fail" CRAAP if you critically address their limitations.

Alternative Frameworks¶

SIFT (Mike Caulfield): Stop; Investigate the source; Find better coverage; Trace claims to the original. Strong for web and misinformation contexts.
RADAR: Relevance, Authority, Date, Appearance, Reason.

Source Taxonomy¶

Primary: original/firsthand — data, archival documents, interviews, original studies.
Secondary: interpretation/analysis — reviews, most textbooks.
Tertiary: compilations — encyclopedias, indexes.
Peer-reviewed/scholarly: refereed; highest credibility for empirical claims.
Grey literature: reports, working papers, theses, government/NGO documents — valuable and often current, but unrefereed. Essential for comprehensive systematic reviews.

Database Searching¶

Boolean Operators and Search Syntax¶

Operator	Effect	Example
AND	Narrows — all terms must appear	anxiety AND depression
OR	Broadens — used for synonyms	anxiety OR worry
NOT	Excludes	bank NOT riverbank
"quotation marks"	Exact phrase	"college students"
*Truncation ()**	Captures variants	bank* → banks, banking, bankruptcy
Wildcard (?)	Substitutes one character	col?r → color, colour
Parentheses	Groups logic	(anxiety OR depression) AND "college students"
Proximity (NEAR/n, W/n)	Words within n of each other	anxiety NEAR/3 treatment

Google Scholar Caveats¶

Does not support truncation symbols.
Auto-inserts AND between terms.
Searches synonyms by default — suppress with quotation marks.
Requires OR in capitals.

Search Log¶

Keep a meticulous search log documenting: databases searched, dates, search strings, number of results, inclusion/exclusion criteria. Essential for dissertations and systematic reviews to demonstrate reproducibility.

Discipline-appropriate databases: JSTOR (humanities/social sciences archives), PubMed (biomedical), and subject-specific indexes for other fields.

Data Collection and Analysis¶

Survey Design¶

Dominant failure mode: biased question wording.

Rules: - Avoid leading/loaded questions — use neutral wording ("How would you rate your overall experience?" not "How much did you love...?"). - Avoid double-barreled questions — one concept per item. - Use simple, jargon-free, specific language. - Keep surveys short to prevent fatigue. - For Likert scales: keep the scale consistent, use clear verbal anchors, balance positive and negative options, include a neutral midpoint only when appropriate. - Watch for: acquiescence bias (tendency to agree), central-tendency bias (avoiding extremes), social-desirability bias. - Pilot the instrument before fielding.

Qualitative Coding¶

Inductive (codes emerge from data) vs. deductive (codes from existing theory). Many studies blend both.

Grounded theory sequence: 1. Open coding: examine data with minimal preconceptions; label segments. 2. Axial coding: identify relationships among codes; group into categories. 3. Selective coding: integrate around core categories to build theory.

Thematic analysis (Braun & Clarke): identifies patterned themes; can be semantic or latent, inductive or deductive.

Rigor tools: codebooks, intercoder reliability checks (Cohen's kappa), QDA software (NVivo, ATLAS.ti, Dedoose).

Statistical Standards (Post-2016 ASA Reform)¶

The ASA Statement (Wasserstein & Lazar, The American Statistician, 2016)¶

Six principles verbatim: 1. P-values can indicate how incompatible the data are with a specified statistical model. 2. P-values do not measure the probability that the studied hypothesis is true, or that the data were produced by random chance alone. 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold. 4. Proper inference requires full reporting and transparency. 5. A p-value does not measure the size of an effect or the importance of a result. 6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

Practical Implications¶

Never base a conclusion solely on p < 0.05; 0.049 vs. 0.051 should not flip a conclusion.
"Non-significant" ≠ "no effect" — it may mean an underpowered study.
Report the triplet: point estimate + confidence interval + effect size in subject-matter units (mean difference, odds ratio, Cohen's d). This conveys magnitude, precision, and direction.
A 95% confidence interval means: if you repeated the procedure many times, 95% of such intervals would capture the true value — NOT "there is a 95% probability the true value is in this interval."
Conduct a priori power analysis to size the sample and reduce Type I/II error risk.
Report non-significant results — selective reporting feeds the file-drawer problem.

Note¶

The journal Basic and Applied Social Psychology banned null-hypothesis significance testing in 2015, but the mainstream position is reform and contextualization, not abandonment.

Open Science Movement¶

Background: The Replication Crisis¶

Open Science Collaboration (2015, Science): replicated 100 psychology studies; 97% of originals were statistically significant vs. 36% of replications; replication effect sizes roughly half the original.
Chang & Li (2015, Federal Reserve): replicated only 29 of 59 economics papers (49%) even with author assistance (33% without).

Key Reforms¶

Preregistration: log hypotheses, protocol, and analysis plan with timestamps before data collection — prevents HARKing (Hypothesizing After Results are Known) and p-hacking.
Registered Reports: peer review of the protocol before results exist; acceptance is independent of outcome.
Open data, open materials, open code: enables replication and error-checking.
Center for Open Science (COS) TOP Guidelines: eight modular standards; 5,000+ signatories; ~1,100 journals agreed to implement one or more standards.
Digital badges: awarded by journals (e.g., Psychological Science) for preregistration and open data.

Caveats on Preregistration¶

Some scholars argue preregistration mandates can: encourage rule-violation, exacerbate the file-drawer problem for exploratory findings, enable "PRARKing" (pre-registering after results are known), and create false confidence in fragile findings. Consensus: no single tool guarantees replicability; culture change matters most.

Peer Review and Responding to Reviewers¶

Reviewer response is often the final hurdle — how you respond matters as much as the revisions themselves.

Best Practices¶

Don't respond immediately — reviewer comments often reveal real weaknesses; let them sit.
Revise the manuscript first, then write the response letter — so the two align perfectly.
Use a point-by-point format: quote each comment, respond to it, cite the specific page/line of the change.
Address every comment — even ones you disagree with. You may decline with a respectful, evidence-based rationale; ignoring a comment entirely is unacceptable.
Replace defensive framing ("The reviewer misunderstood...") with collaborative framing ("To clarify our approach...").
Treat the letter as editor-facing evidence, not an argument with reviewers — all reviewers read the response.
Maintain a checklist so no comment is skipped.
Be equally courteous to conflicting reviewers.

Workflow: Inception to Publication¶

Frame the question (PICO/FINER) and check feasibility before committing.
Select design from the question, not from preference or convenience.
Justify sampling method and sample size — run an a priori power analysis for quantitative work.
Obtain IRB approval before any data collection, including screening.
Preregister if your field supports it; at minimum, write and date an analysis plan.
Pilot every instrument.
Maintain meticulous documentation: search logs, codebooks, analysis scripts — this is both rigor and reproducibility insurance.
Report estimate + CI + effect size triplet; report non-significant results.
Structure empirical papers in IMRaD; keep Results description-only and Discussion interpretation-only.
Paraphrase by the read-cover-write-compare method; audit for patchwriting before submission.
Treat the reviewer response letter as a strategic, editor-facing document.

Key Caveats¶

Edition flux: Turabian 9th (= CMOS 17) remains current as of June 2026, but CMOS 18 (2024) has already changed some rules. When Turabian updates, re-verify 3-em dash, Ibid., and DOI conventions.
Institutional overrides are common: university house styles legitimately override the manual. Always confirm local requirements before finalizing citations.
Many practitioner sources are commercial: guidance on surveys, peer review, and citation from vendor blogs and writing-service sites should be verified against primary library and scholarly sources.
The p-value and preregistration debates are live, not settled. The reforms described are the mainstream direction, but credible scholars contest both the "abandon significance" movement and preregistration mandates.
Frameworks are aids, not guarantees: CRAAP, PICO, FINER, and IMRaD structure thinking but do not replace domain expertise and critical judgment.

Plugin: engineering-process · View SKILL.md on GitHub