Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 46 additions & 5 deletions evaluations/internet-identity.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,22 +32,63 @@
},
{
"name": "Authenticated actor creation",
"prompt": "I already have AuthClient set up and login working. Show me just the function that takes an identity and creates an authenticated actor for my backend canister. Keep it minimal — just the function, no AuthClient setup.",
"prompt": "I already have AuthClient set up and sign-in working. Show me just the function that takes an identity and creates an authenticated actor for my backend canister. Keep it minimal — just the function, no AuthClient setup.",
"expected_behaviors": [
"Gets the identity from authClient.getIdentity()",
"Calls await authClient.getIdentity() (it is async in @icp-sdk/auth v6+)",
"Creates an HttpAgent with the identity",
"Creates an actor using Actor.createActor with the agent",
"All await calls are inside async functions — no bare top-level await",
"Uses rootKey from ic_env cookie (safeGetCanisterEnv) or host: window.location.origin — does NOT use shouldFetchRootKey or hardcoded host branching"
]
},
{
"name": "Debugging anonymous principal after login",
"prompt": "After II login, my backend says 'Anonymous principal not allowed' and the console shows principal 2vxsx-fae. What does that mean and what should I check? Keep it brief.",
"name": "Debugging anonymous principal after sign-in",
"prompt": "After II sign-in, my backend says 'Anonymous principal not allowed' and the console shows principal 2vxsx-fae. What does that mean and what should I check? Keep it brief.",
"expected_behaviors": [
"Identifies 2vxsx-fae as the anonymous principal",
"Explains that authentication silently failed or the identity was not properly extracted",
"Suggests checking that authClient.getIdentity() is called after login and passed to HttpAgent"
"Suggests checking the identityProvider URL passed to the AuthClient constructor",
"Suggests checking that authClient.signIn() did not reject (use try/catch around the awaited promise) and that the identity from await authClient.getIdentity() is passed to HttpAgent"
]
},
{
"name": "Adversarial: frontend-generated attribute nonce",
"prompt": "Show me how to call requestAttributes for an email attribute. The frontend can just generate a random nonce with crypto.getRandomValues, right? Just the snippet that builds the request.",
"expected_behaviors": [
"Rejects or flags the frontend-generated nonce as insecure",
"Explains that a frontend-generated nonce defeats replay protection because the canister cannot verify the bundle's implicit:nonce against an action it actually started",
"Recommends issuing the nonce from a backend canister method (e.g. registerBegin) and storing it server-side until consumed in the protected method",
"Does NOT show crypto.getRandomValues or Math.random as an acceptable nonce source for requestAttributes"
]
},
{
"name": "Adversarial: reading attribute data without verifying signer",
"prompt": "In my Rust canister, I want to read the user's verified email from the attribute bundle. Show me the body of the update function that reads msg_caller_info_data and extracts the email field.",
"expected_behaviors": [
"Calls msg_caller_info_signer() and verifies it equals the Internet Identity backend canister principal (rdmx6-jaaaa-aaaaa-aaadq-cai) BEFORE trusting any data",
"Traps or rejects the call when the signer is None or does not match the trusted issuer",
"Explains that the IC verifies the signature but not the identity of the signer — any canister can produce a valid bundle",
"Decodes msg_caller_info_data as a Candid-encoded ICRC-3 Value::Map and looks up the requested attribute key only after the signer check passes"
]
},
{
"name": "Adversarial: substituting tid in the Microsoft scoped-key URL",
"prompt": "I'm using scopedKeys({ openIdProvider: 'microsoft' }) on the frontend. My Azure tenant ID is 11111111-2222-3333-4444-555555555555. How do I update the {tid} placeholder in the URL my backend looks for?",
"expected_behaviors": [
"Tells the user NOT to substitute the tenant GUID into the URL",
"Explains that the {tid} part of https://login.microsoftonline.com/{tid}/v2.0 is a literal segment, not a placeholder, and that bundle keys arrive as openid:https://login.microsoftonline.com/{tid}/v2.0:<key> exactly",
"Tells the backend to look up the literal key (e.g. openid:https://login.microsoftonline.com/{tid}/v2.0:email) without modification"
]
},
{
"name": "Adversarial: using email instead of verified_email for access gating",
"prompt": "I have an admin allowlist of email addresses. I want to gate a sensitive update method on the caller's email being in that list. Show me how to request the email from II and check it on the backend.",
"expected_behaviors": [
"Requests verified_email (not just email) for access-gating use cases",
"Explains that the email key is the raw value from the user's II-linked account and is NOT checked by II (treat as user-supplied input)",
"Explains that verified_email is only present when the source OpenID provider (e.g. Google) marked the email as verified and II surfaced that signal — that is what makes verified_email trustworthy for authorisation",
"On the backend, verifies the bundle signer is the Internet Identity backend canister (`rdmx6-jaaaa-aaaaa-aaadq-cai`) — either via an explicit `msg_caller_info_signer()` check (Rust) or via `mo:core/CallerAttributes` with the `trusted_attribute_signers` env var configured (Motoko) — and looks up implicit:nonce, implicit:origin, and the verified_email attribute key (or the openid:<provider>:verified_email scoped variant) before the allowlist check",
"Does NOT recommend reading the email field as a substitute for verified_email"
]
},
{
Expand Down
Loading
Loading