Appendix A — Working with AI

An AI assistant is now part of how most students write code, so this appendix suggests one way to use one while working through this book. The short version is that you attempt the task first, and the assistant then evaluates what you produced. That order matters, because an assistant that writes your script for you leaves you unable to tell whether the output is right, and telling whether output is right is most of what this book teaches. The workflow below walks a script you already wrote through eight checks. The sections after it explain the ways an assistant will mislead you and how to catch that. Exercise 1 of every chapter points back here, so you may work through the checks many times.

Whatever your own course requires takes precedence over anything suggested here.

A.1 Ground Rules

The suggestion is that you attempt the task first, and the assistant then evaluates what you produced. Asking for code you have not tried to write, or pasting an exercise prompt and submitting whatever comes back, skips the part where the learning happens. Either way, you stay the judge of every flag the assistant raises, which is what the eight steps are built around.

What to Upload.

The scripts in this book use built-in R datasets, which contain no personal data, so uploading them raises no issues. Other data is worth more thought: data about identifiable people, licensed data you do not own, and another student’s work are all things to keep out of a chat window. If you are unsure whether a dataset is safe to paste, it is better to ask before pasting than after.

Disclosing What You Did.

Step 8 suggests recording, inside the script itself, which flags you accepted and which you rejected. A course that asks you to disclose AI use is usually asking for something like this record, and it is the part that shows the judging was yours. A log with no rejections in it is worth a second look, since it usually means you accepted everything without checking.

A.2 Reviewing Your Script

Step 1: Comment Your Code.

Open the script you wrote while reading the chapter and comment it in the style shown in Comments. Each comment should say what the block does and why you ran it. Do this before you open an assistant, and a line you cannot describe in plain English is a line you do not yet understand.

Everything after this step happens in the copy, so that the difference between the two files is a record of what you changed.

  • Save your work as Script_01.R and do not edit it again.
  • Create a copy, Script_01_revised.R, saved next to the original in the same folder Desktop/ECON_XXXX/Code.

Update the names for later chapters. E.g., use Script_02.R for chapter 2.

Step 2: Predict, Then Run.

Pick your two or three main results and write down, as comments, what you expect before you run anything. A number you write down after seeing the output is not a prediction.

Restart R with Session > Restart R, and check that the Environment pane is empty. Then use the menu beside Source to select Source with Echo, which runs the whole script and prints each command and result. Do not use Run, which executes only the current line or selection.

If it errors, fix the error, then restart and Source again.

If you are stuck, upload the script and the error message.

Here is my R script and the error message it produces. What is causing the error, and what should I look at to fix it myself? Do not give me corrected code.

Once it runs from start to finish, compare each result against what you predicted.

Step 3: Ask Whether Your Comments Match Your Code.

Ask whether your comments are accurate and you stay the judge.

Here is an R script from an introductory statistics course, with my own comments. For each comment, tell me whether it accurately describes what that line of code actually does. Quote the comments that are wrong or incomplete, and explain the mismatch. Do not rewrite my comments or my code. Do not suggest replacing my method and keep to the notation used in my script.

The last line matters because this book builds inference from simulations, and an assistant left to itself steers you toward canned test functions and the notation of other textbooks. Settle each flag by running the line and doing a small hand-check. (Look at what comes back, since assistants write confidently when they are wrong.)

Step 4: Ask Whether You Estimated the Right Quantity.

For each main result, write one sentence naming the quantity you meant to estimate, in words and in the notation of the chapter. Put those sentences under the script and ask.

Here is an R script, followed by one sentence per result naming the quantity I meant to estimate. For each, tell me whether the code computes that quantity or a different one, and name the difference. Do not rewrite my code.

You are the judge about what you want, and whether the code is doing what you want.

Step 5: Write Your Own Summary and Have It Critiqued.

At the top of the script, write three sentences on the main statistical ideas. (This is what the chapter is about, not what the code does technically.)

Then start a new AI chat and ask

Here are three sentences I wrote from memory about a chapter in my introductory statistics course, followed by the chapter’s own summary. […] Tell me which of my statements are wrong, which are too vague to be useful, and what important idea I left out. Do not write a corrected summary for me.

Step 6: Ask Your Own Questions.

Now ask the AI assistant about whatever you found hardest, in your own words. A vague question gets a vague answer, so name the specific idea, number, or line that lost you.

I do not follow X. Explain it a different way, then ask me three questions to check whether I understand it.

Questions about code work the same way, as long as they are specific.

Assuming Y already exists, why does Y < -43 not assign the value 43, when Y <- 43 does?

Finish by having the assistant quiz you, since answering a question tests your understanding better than reading an answer does.

Step 7 (Optional): Reorganizing Your Script.

Once the revised script is saved, you can ask an assistant to reorganize it.

Help me reorganize the code into coherent blocks, for easy recall later.

Ask only when you already understand the script and want it tidier, since a long plain script is usually easier to learn from than a compact one built out of functions.

Step 8: Save Your Work.

Just after your big-picture summary at the top of the revised script, record what you did with each flag. The rejections are useful to show that you did the judging.

# --- AI review ---
# Accepted:
#   ~ line 22: comment said "mean", code computed the median. Comment fixed.
#   ~ line 41: bootstrap resampled rows, not students. Now resamples by student.
# Rejected:
#   ~ line 12: claimed na.rm=TRUE hid a problem. Checked sum(is.na(x)); there are none.
#   ~ line 58: suggested t.test over the permutation test. Not the method this chapter builds.

A.3 When the Assistant Is Wrong

An assistant produces text that is likely to follow your prompt, which is not the same as text that is true. It cannot run your code, check a number, or see what your data contain, so a wrong answer arrives in exactly the same confident tone as a right one. The fix is not to distrust everything, but to check the small number of claims that actually matter, which is usually quick.

A Worked Check.

Suppose you asked whether your comments match your code, and the assistant returned this flag.

Line 4 is wrong. sd() computes the population standard deviation, dividing the sum of squared deviations by \(n\). Your comment calls it the sample standard deviation.

That is a claim about arithmetic, so settle it with arithmetic.

Code 
X <- USArrests[,'Murder']
n <- length(X)

# The two candidate formulas, computed by hand
sqrt(sum((X - mean(X))^2) / n)        # divides by n
## [1] 4.311735
sqrt(sum((X - mean(X))^2) / (n - 1))  # divides by n-1
## [1] 4.35551

# What R actually does
sd(X)
## [1] 4.35551

sd(X) matches the second number, so R divides by \(n - 1\) and the flag is wrong. Your comment stays, and the rejection goes into the Step 8 log.

# Rejected:
#   ~ line 4: claimed sd() divides by n. Hand-checked both formulas; sd() matches n-1.

The check took three lines and one comparison, which is the general shape of it. Any claim about what a function computes can be settled by computing it both ways and looking.

Ask an assistant whether var() divides by \(n\) or by \(n - 1\), then settle it by hand before accepting the answer.

Code 
X <- USArrests[,'Murder']
n <- length(X)

sum((X - mean(X))^2) / n        # divides by n
## [1] 18.59106
sum((X - mean(X))^2) / (n - 1)  # divides by n-1
## [1] 18.97047
var(X)
## [1] 18.97047

Which line does var(X) match, and is that consistent with what you found for sd(X) above?

Failure Modes to Expect.

Four patterns come up often enough to be worth naming.

Substituting the standard method. You build a bootstrap or a permutation test, and the assistant proposes t.test() instead. It is not wrong that t.test() exists, but it is wrong as a review of what you wrote, because this book builds inference from simulation on purpose. Tell the assistant which method the chapter uses, and reject the flag if it ignores you.

Switching notation. The assistant writes \(\bar{x}\) and \(s\) where this book writes \(\hat{M}\) and \(\hat{S}\), or renames your objects to match another textbook. Notation is the reference for what this book uses.

Inventing arguments. Plausible-looking arguments that do not exist, such as a prob= argument where the function actually takes freq=. Reading ?hist settles this immediately.

Misstating what a result means. The most common single error is describing a \(p\)-value as the probability that the null hypothesis is true. It is not, and Hypothesis Tests explains what it is instead.

A.4 Writing Good Prompts

The prompts in the eight steps share four features, which you can reuse when writing your own.

Name the specific thing. “I do not follow the bootstrap” gets you a lecture, while “I do not follow why line 22 resamples rows instead of students” gets you an answer.

Say what you do not want. Every prompt above ends with a restriction, such as “Do not give me corrected code.” Without it you get a rewritten script and learn nothing.

State the method. Say that you are building inference from simulations, or the assistant will steer you toward canned test functions.

Ask to be tested. “Ask me three questions to check whether I understand it” is worth more than another paragraph of explanation, because answering a question tests your understanding and reading an answer does not.

Further Reading.

  • Korinek (2023) – a survey of where these tools help and where they fail in economic research.
  • Data Scientism – how confident-looking results mislead, which is the same problem in a different form.
Korinek, Anton. 2023. “Generative AI for Economic Research: Use Cases and Implications for Economists.” Journal of Economic Literature 61 (4): 1281–317. https://doi.org/10.1257/jel.20231736.