add cs350 fall 2015 midterm #88

qimcis · 2026-01-18T04:43:54Z

Description

Add CS350 Fall 2015 Midterm exam data and solution file to the repository under the courseexam benchmark.

Changes

Add exam files to /home/qi/system-intelligence-benchmark/benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm.
Add solutions file F15-midterm-soln.pdf for the CS350 Fall 2015 Midterm.
Add exam metadata (title: CS350 Fall 2015 Midterm, exam ID: cs350_fall_2015_midterm) and note that there are no reference files.

Testing

Ran python3 courseexam/prepare.py to validate intake and preparation of the added exam data.

Checklist

Tests pass locally
Code follows project style guidelines
Documentation updated (if needed)

tareknaser · 2026-01-19T16:41:52Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+  V(sc);
+}
+
+Re-implement func1 and func2 (including any necessary global variable declarations and initialization) using locks and condition variables (no semaphores). Your re-implemented functions must have the same behavior as the original semaphore-based functions.


This question was modified from the original exam. It is missing some information and guidelines regarding the solution

tareknaser · 2026-01-19T16:42:11Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+Global variables / Initialization
+struct semaphore *sa;
+struct semaphore *sb;
+struct semaphore *sc;
+sa = semcreate("A",1);
+sb = semcreate("B",1);
+sc = semcreate("C",0);
+
+void func1(){
+  P(sa);
+  funcA();
+  V(sa);
+  P(sc);
+}
+
+void func2(){
+  P(sb);
+  funcB();
+  V(sb);
+  V(sc);
+}


We should probably include this as a code block

tareknaser · 2026-01-19T16:43:41Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+  "points": 12,
+  "type": "Freeform",
+  "tags": ["concurrency", "synchronization", "locks", "condition-variables"],
+  "answer": "A correct re-implementation uses separate locks for protecting funcA and funcB critical sections and a lock+condition variable to implement the signaling behavior of sc, preserving ordering so that func1 waits until func2 signals. For example: declare locks la and lb for funcA and funcB respectively; declare a lock lc, a condition variable cv, and an integer count initialized to 0. func1 acquires la, calls funcA, releases la, then acquires lc, while(count<=0) cv_wait(cv, lc); count--; release lc. func2 acquires lb, calls funcB, releases lb, acquires lc, count++; cv_signal(cv, lc); release lc. (Any equivalent correct implementation that preserves semantics earns full credit.)",


There is a reference implementation in the answer sheet that we can use

tareknaser · 2026-01-19T16:45:15Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+## Question 2a [2 point(s)]
+
+What is the maximum number of entries in a page table in this system?
+
+```json
+{
+  "problem_id": "2a",
+  "points": 2,
+  "type": "Freeform",
+  "tags": ["paging", "virtual-memory"],
+  "answer": "16",
+  "llm_judge_instructions": "Award 2 points for the answer '16'. 0 points for any other answer."
+}
+```


The question is missing essential information

tareknaser · 2026-01-19T16:50:07Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+Consider the function:
+
+void bump(){
+  int y;
+  x++; // x is a global variable
+  y = x;
+  kprintf("%d", y); // print value of y
+}
+
+Assume x is a volatile global integer initialized to 0 before any calls to bump. Suppose the program uses k concurrent threads, each calling bump once. Calls to kprintf are atomic (i.e., prints do not interleave). For k = 4 running on a single-core processor, which of the following outputs are possible? For each output, write "yes" if possible and "no" if not.
+


There is some missing information from the question text

tareknaser · 2026-01-19T16:50:48Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+  "type": "Freeform",
+  "tags": ["concurrency", "multithreading"],
+  "answer": "Possible: 1234 (yes), 4321 (yes), 2222 (yes), 4444 (yes). Not possible: 0123 (no), 1235 (no), 012 (no), 1124 (no).",
+  "llm_judge_instructions": "Part a (6 pts): Award 1 point each for the following six correct determinations: 1234 -> yes (1 pt), 4321 -> yes (1 pt), 2222 -> yes (1 pt), 4444 -> yes (1 pt), 0123 -> no (1 pt), 1235 -> no (1 pt). (The other outputs were included for completeness but will not be graded for points in this part.)"


The question includes some information on how to score ("Note that you must get at least half of these correct to receive any credit for your answer.")

tareknaser · 2026-01-19T16:51:11Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+## Question 3b [2 point(s)]
+
+Suppose instead the concurrent program (with k = 4) runs on a machine with two single-core processors. Do your answers from part (a) change? If not, write "No Change". If so, indicate one output string from part (a) for which you would give a different answer and explain briefly.


This question seems to be a continuation of the previous one. Each question should be fully standalone. See the guidelines on handling multi-part question

tareknaser · 2026-01-19T16:53:43Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+Virtual Address -> Physical Address
+0x0000 -> 0x2224
+0x0007 -> 0x3234
+0x0008 -> 0x3224
+0x0010 -> 0x1AF0
+0x0018 -> 0x3234
+0x0008 -> 0x0AF0


The table content seems to be different than the exam PDF. Could you double check?

tareknaser · 2026-01-19T16:54:54Z

benchmarks/courseexam_bench/data/raw/cs350_fall_2015_midterm/exam.md

+a. Given the translations shown, is it possible that the MMU used dynamic relocation (i.e., a single relocation register) to translate P’s virtual addresses to physical addresses? If so, write "YES" and indicate the relocation register value. If not, write "NO" and briefly explain why not.
+
+b. Is it possible that the MMU used paging with page size 64 KB (2^16 bytes)? If so, write "YES". If not, write "NO" and briefly explain why not.
+
+c. Is it possible that the MMU used paging with page size 4 KB (2^12 bytes)? If so, write "YES". If not, write "NO" and briefly explain why not.
+
+```json
+{
+  "problem_id": "4",
+  "points": 8,
+  "type": "Freeform",
+  "tags": ["memory-management", "mmu", "virtual-memory"],
+  "answer": "a) NO — the translations imply different relocation offsets for different addresses, so a single relocation register cannot explain them. b) NO — with 64 KB pages the low 16-bit offsets must match between virtual and physical addresses on the same page, but observed offsets differ. c) YES — 4 KB pages are consistent with the observed mappings (same 12-bit offsets where required).",
+  "llm_judge_instructions": "Total 8 pts: Part a (3 pts) — award 3 pts for correctly answering NO with a brief correct justification that translations use different offsets. Part b (3 pts) — award 3 pts for correctly answering NO with a brief correct justification about 16-bit offsets. Part c (2 pts) — award 2 pts for correctly answering YES with brief justification. Partial credit allowed for partially correct explanations."
+}


I think some information is missing from the questions

also the answers do not match the reference answers in the exam PDF

github-actions · 2026-01-19T18:02:51Z

Experimental courseexam benchmark run

Exams tested:

cs350_fall_2015_midterm

Evaluation file: Download the evaluation-results artifact from this workflow run and inspect it using inspect view <file>.eval or the Inspect VS Code extension. For more details on how to use the web-based log viewer, see Inspect Log Viewer documentation

This experimental run helps you verify that questions have enough context for the LLM to answer and that grading rubrics are appropriate.

qimcis added 4 commits January 17, 2026 11:55

add CS350 Fall 2015 Midterm

f8d80a5

add CS 350 Fall 2015 Midterm

c01156c

add CS350 Fall 2015 Midterm

dfb42df

add CS350 Fall 2015 Midterm

6811020

tareknaser requested changes Jan 19, 2026

View reviewed changes

sys-intelligence deleted a comment from github-actions bot Jan 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add cs350 fall 2015 midterm #88

add cs350 fall 2015 midterm #88

Uh oh!

qimcis commented Jan 18, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

tareknaser Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		## Question 3b [2 point(s)]

		Suppose instead the concurrent program (with k = 4) runs on a machine with two single-core processors. Do your answers from part (a) change? If not, write "No Change". If so, indicate one output string from part (a) for which you would give a different answer and explain briefly.

add cs350 fall 2015 midterm #88

Are you sure you want to change the base?

add cs350 fall 2015 midterm #88

Uh oh!

Conversation

qimcis commented Jan 18, 2026

Description

Changes

Testing

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 19, 2026

Experimental courseexam benchmark run

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants