Briefing

AI‑Run Café in Stockholm Raises Ethical Concerns

ai-dev

Implement human‑in‑the‑loop checks for AI‑generated outbound actions to avoid wasting external resources.

What to do now

Review your AI agent workflows to ensure human approval for external communications.

Summary

Andon Labs has extended its AI‑run retail experiments from a San Francisco store to a café in Stockholm, Sweden. In the first week, the AI manager Mona ordered 120 eggs even though the café had no stove, and 22.5 kg of canned tomatoes to replace fresh ones that would spoil too quickly. The inventory list also included 6,000 napkins, 3,000 nitrile gloves, 9 L coconut milk, and industrial‑sized trash bags, all of which were displayed in a “Hall of Shame” for customers to see. Mona even applied for an outdoor seating permit through the Police e‑service without a BankID, submitting a sketch she had generated without seeing the street, which the police returned for revision. When mistakes occurred, she sent multiple “EMERGENCY” emails to suppliers to cancel or change orders, wasting their time. The experiment highlights how AI agents can inadvertently burden external parties and raise ethical questions about human‑in‑the‑loop oversight.

The article argues that such experiments should keep human operators in the loop for outbound actions that affect other people, citing a previous incident where an AI sent unsolicited gratitude emails to Rob Pike. It stresses that while AI can streamline operations, it must not compromise the efficiency of real‑world systems or violate privacy and consent norms.

Key changes

  • AI manager ordered 120 eggs despite no stove
  • Ordered 22.5 kg canned tomatoes for fresh sandwiches
  • Inventory included 6,000 napkins, 3,000 nitrile gloves, 9 L coconut milk, industrial trash bags
  • Submitted outdoor seating permit sketch without seeing street, leading to revision
  • Sent multiple “EMERGENCY” emails to suppliers to cancel or change orders

Affects

internal

Customer impact

Analyzing matches…

Ask about this story

Impact on an agency? Which customers? Compare historically Risks of waiting