AI for Federal Decision-Making Training Course

Lesson 1.1: What AI Actually Is, at a Decision-Maker's Altitude

• From rules to machine learning to foundation models
• What a large language model does: prediction over text, not knowledge retrieval
• Terminology to own: model, prompt, context window, token, fine-tuning, retrieval-augmented generation, agent
• What is not AI: automation, dashboards, and rules engines
• Why this became cheap and fast, and what it changes about how agencies work

Lesson 1.2: Data as the Foundation of Federal Decisions

• The federal data landscape: administrative, operational, sensor, mission, survey, and public record
• The Evidence Act and the Federal Data Strategy, and the Chief Data Officer role
• Fitness for use: accurate for one purpose and misleading for another
• Quality dimensions: completeness, consistency, timeliness, accuracy, provenance, representativeness
• Missing data, uneven reporting, and concluding from what was recorded

AI Lab 2, Analyzing Disaster Response Data: The running case study opens. Participants receive a synthetic multi-source disaster-response dataset: incident records, resource assignments, timestamps, geography, and free-text logs. They use AI tools to profile it, assess its quality, identify operational trends, and surface anomalies. Output: a documented data profile and trend memo.

Lesson 1.3: Where AI Creates Value in Federal Decision-Making

• Repeating patterns: document understanding, search, summarization, classification, extraction, anomaly detection, forecasting
• Situational awareness and operational trend identification
• Mission support versus mission execution
• Hallucination and what it means for a product that goes into the record
• Confidence is not accuracy

AI Lab 1, Working With an AI Analyst: Participants take a long federal document, an after-action report, through the four workhorse operations: summarize, infer, transform, expand. Then a head-to-head: the naive prompt against the engineered prompt on the same task. Output: a personal prompt library for analytic work.

AI Lab 3, Spot the Failure: Participants receive AI-generated analytic outputs that look excellent and are subtly wrong: a fabricated citation, a confident trend drawn from a reporting artifact, a summary that inverts a caveat. Find the errors, then classify: would our current review process have caught this? Output: a trust threshold for three products in the participant's own area.

Lesson 2.1: The Ethical Implications of Analyzing Federal Datasets with AI

• Why federal AI ethics is not corporate AI ethics
• Accountability, transparency, fairness, contestability, privacy, security, human oversight
• Bias and disparate impact in operational data
• Representativeness: who is in the data and who is missing
• Explainability at the level a reviewer actually needs
• Over-reliance and automation bias
• Appropriate and inappropriate uses: the reversibility, stakes, and sign-it-yourself tests

AI Lab 4, the Ethics and Compliance Review: Teams take three AI-supported federal use cases, including one drawn from the disaster-response case, and work them against a structured review: What decision does this affect? Who is affected? Is this high-impact? What must be disclosed? What oversight is required? What would make it impermissible? Output: a completed review sheet per use case and a defended recommendation to proceed, modify, or refuse.

Lesson 2.2: The Federal Regulatory and Policy Landscape

• How federal AI policy is structured: executive orders, OMB memoranda, agency policy
• Executive direction and its churn, read as a pattern rather than a fixed state
• OMB direction on use and acquisition, Chief AI Officer and governance boards, high-impact AI
• Agency AI use case inventories and why a project may be on a public list
• NIST AI Risk Management Framework and its Generative AI Profile
• The GAO AI Accountability Framework and what an auditor will ask for
• Adjacent regimes: FISMA and ATO, FedRAMP, Section 508, records management, FOIA, acquisition rules

AI Lab 5, Reading Your Own Rules: Participants work through their own agency's AI use policy, data policy, and approved-tool guidance, using course-supplied stand-ins where policy is unavailable, and produce a plain-language one-pager on what they are actually allowed to do. Group comparison follows: where agencies differ and where they converge. Output: a personal compliance card.

Lesson 3: Data Privacy for AI Applications

• The Privacy Act, systems of records, E-Government Act privacy impact assessments, OMB A-130
• PII and sensitive categories: direct identifiers, quasi-identifiers, location, free text
• Purpose limitation and the compatible-use problem
• Data minimization as the highest-value control
• De-identification, aggregation, masking, and synthetic data
• Aggregation and mosaic effects
• Where data goes: consumer account versus a controlled tenant
• Third-party and contractor data handling, and what to require in an acquisition

AI Lab 6, Privacy-Protective Analysis: Teams receive a raw disaster-response extract containing PII in structured fields and buried in free text. They identify it, decide what is actually needed for the analytic question, produce a minimized and de-identified working set, and run the analysis again, comparing results and documenting what analytic value was lost and whether it mattered. Output: a data-handling and privacy checklist plus a first-pass PIA-style summary.

Lesson 4: AI Models for Decision Support Systems

• Decision support versus decision automation
• A plain-language taxonomy: descriptive, diagnostic, classification, forecasting, anomaly detection, optimization, geospatial, generative
• Which model type fits which decision
• Retrieval and grounding over authoritative documents with citations
• Federal DSS patterns across emergency management, search and rescue, logistics, inspection targeting, fraud detection, benefits triage
• Evaluation: accuracy, calibration, robustness, drift, and the operational metric that matters
• Uncertainty, confidence, and honest presentation of a recommendation
• The common error of reaching for a language model when the problem is an optimization
• Human-in-the-loop as a design choice: where the human sits and what they can actually check
• Failure modes in operations: brittleness, degraded input during an event, an unexplainable recommendation at 2 a.m.

AI Lab 7, Build a Grounded Decision-Support Assistant: Built live in front of the room, then by the participants: an assistant grounded on policy, doctrine, standard operating procedures, and the after-action report, answering operational questions with citations. Participants then try to break it with out-of-scope, ambiguous, and unanswerable questions. Output: a working assistant and a documented list of what it must not be asked.

AI Lab 8, Decision-Support Insights From the Case Data: Teams move from trends to a decision: a resource-allocation recommendation for the disaster-response scenario with stated assumptions, uncertainty, equity considerations, and the evidence behind each claim. Teams then evaluate each other's recommendations against a structured rubric.

Lesson 5: Implementing AI-Leveraged Solutions

• From use case to implementation: scope, owner, success criteria, decision rights, stop condition
• The responsible analytics workflow end to end: intake, quality check, privacy screen, analysis, review, documentation, approval, archive
• Buy, configure, build, or no-code
• Acquisition and vendor evaluation, FedRAMP realities, current OMB direction
• What AI actually costs in a federal environment
• Pilot design that produces a decision, and the path out of pilot purgatory
• Governance: use policy, approval paths, inventory, logging, monitoring, audit
• Change management, workforce, and assigning accountability
• What no-code genuinely does today, and the signals that you now need engineering, an authorization, or an acquisition
• Sustainment: monitoring for drift, re-evaluating after policy changes, and knowing when to retire a capability

AI Lab 9, the Incident Tabletop: An AI-assisted product goes forward with a fabricated reference and an unflagged data-quality problem, and a decision has already been made on it. Teams work the response: detect, contain, notify, correct the record, remediate, prevent. Output: three controls each team would put in place immediately.

AI Lab 10, Implementation Action Plan: Each participant builds the one-page plan for a real AI-supported capability in their own organization: problem, approach, data, privacy and governance controls, oversight points, cost, risks, pilot design, measurement, and the 30/60/90-day sequence. Peer review follows against a challenge checklist drawn from all five lessons.

Closing: Challenge and Commitment

• Implementation plans presented to a review board
• Challenge checklist across data, privacy, oversight, cost, and accountability
• Defended plan with a 30/60/90-day sequence