Bazak AI Shopping Copilot

This project is a home assignment from Bazak. It lets visitors search for products using AI, through a chat experience. You type what you're looking for in plain language, and the assistant finds matching products for you.

Setup & Run

Requires Node 20.9+.

npm install
cp .env.example .env   # then set OPENAI_API_KEY=sk-...
npm run dev

The server is running on port 3000.

Multiple types of tests for different boundaries within the app:

npm run test            # Vitest: unit + integration, offline, no API key
npm run build           # type-check + production build

# Playwright (one-time browser install)
npx playwright install chromium
npm run test:e2e:smoke  # Playwright: UI with /api/chat mocked (no key)

npm run smoke:live      # Playwright: real OpenAI + real DummyJSON (needs key)
npm run eval            # LLM planning eval, live OpenAI calls (needs key)

Architecture & Framework Choice

I started by framing the complexity of the problem. There are no multi-step workflows here that would justify reaching for something heavy like LangGraph or Mastra. So I kept it simple: state lives on the client, and the server is a thin layer whose only job is to forward requests to the AI provider securely, so the API key never reaches the browser.

Vercel AI SDK was the right fit because it gives me freedom over the parts this task is actually about (retrieval, state, validation, failure handling) while handling the commodity parts for me (streaming, tool calls, rendering results inline in the chat).

What I rejected:

• LangGraph / Mastra: built for multi-step agent workflows. This task is a single plan-then-retrieve step, so all that orchestration would be overhead with nothing to show for it.
• From scratch: streaming and tool plumbing are solved problems. Rebuilding them by hand is easy to get subtly wrong and pulls time away from what actually matters here.
• assistant-ui / CopilotKit: they bring their own UI, state, and persistence. I wanted to own those myself, since they're core to this task.

Retrieval Strategy

As soon as the user sends a message, the model gets the last 12 messages plus the products already shown as context, reads the current need, and derives the intent. The intent decides how we search: by category (/products/category/{slug}) when it points at a known category, by keyword (/products/search?q=...) when it's free text, a hybrid of both when it's both, or the full catalog (/products?limit=0) as a fallback when the narrower calls return too little. We also pull the category list (/products/category-list) to match the user's wording to a real category and fix it when it's close but off ("skincare" becomes "skin-care"), and every call uses select so we only fetch the fields we use.

When the intent is clear, the model turns the request into a structured search with the sort order and filters (price, brand, rating). We apply those ourselves, because the catalog's search doesn't support them.

We handle ambiguity by asking instead of guessing. If the request is too open ("something cool"), we ask the user to narrow it down rather than throwing random products at them.

We handle off-catalog requests by turning them down up front. If someone asks to book a flight, the model says it can only help with shopping instead of running a pointless search.

We handle multiple needs in one message by splitting them. "A laptop and headphones" becomes two searches, each shown as its own group.

And when a search comes back empty, we show the closest items we found instead of nothing.

Conversation & State

The entire conversation is persisted on the client, in localStorage. I went this way because:

1. No syncing issues between client and server, where each side ends up with a different version of the conversation.
2. For the scope of a simple chat, it's enough.
3. The conversation is unlikely to hold anything sensitive.
4. I don't expect enough messages to need something heavier like IndexedDB.

I still cared about the edge cases though. If storage fills up I keep the conversation flowing from memory and just let the user know with a notification, and if the saved data ever comes back corrupted or from an older version I don't want it to crash, so it falls back to a clean state and keeps going. Same thing if the user clears storage in the middle of a chat, it carries on from memory and the next save writes it back.

Evaluation

The idea is to test each part at its own boundary, and mock at the right seam. To test my own logic I mock the catalog, so the model isn't even in the picture. To test the UI I mock the model and feed it canned responses, so the rendering is predictable. To test reality I mock nothing.

That gives four layers:

1. Unit / integration: my logic on its own, catalog mocked. Mode selection, filtering, ranking, category repair, the fallback, building context from past messages, and storage recovery.
2. UI with the model mocked: that results actually render as cards, that groups, near-misses and error states show, that off-catalog stays text-only, and that conversations persist and switch.
3. Live UI smoke: the whole thing against the real model and catalog. Off by default, it costs money.
4. Planning eval: whether the model plans the right search from plain language (right category, right sort, splits multiple needs, refuses off-catalog, asks when it's vague). The model isn't deterministic, so I run each case a few times and read the pass rate instead of a hard pass/fail.

The first two gate CI and catch my own regressions. The live ones catch the model drifting. What slips through is whether the picks are actually good (it can return a real but wrong product) and live catalog changes, since most layers mock them.

Known Limitations

Here's where it's weak today, and roughly what I'd change with more time.

1. Persistence is just localStorage, no real database. On top of that, every save rewrites the whole history at once, so it scales with how many conversations you have, not the active one. With more time this moves to a database, with localStorage scoped down to a pointer or an offline cache.
2. Retrieval is keyword-based, not semantic, so descriptive needs ("something for a rainy hike") don't do well. The fix is embedding the catalog and ranking by similarity.
3. The fallback can lose relevance. On a broad query it ends up ranking the whole catalog, so a cheapest-first sort can surface something cheap but unrelated instead of what was actually asked for.
4. Category repair is static. If the catalog renames a category, nothing re-syncs and it quietly drops to a keyword search.
5. Follow-up references like "the second one" or "I like X" are matched loosely, so they can land on the wrong item.
6. No real observability, so when something fails I can't easily see why.
7. Thin memory: only the last ~12 messages and the products already shown go to the model, so a long conversation loses its early context.

And the obvious scope cuts: no streaming resume, so an interrupted reply is lost, and it's discovery only, no pagination, cart, or checkout.