Top
Best
New

Posted by jyunwai 3 hours ago

Mamdani to kill the NYC AI chatbot caught telling businesses to break the law(themarkup.org)
102 points | 20 comments
hashberry 8 minutes ago|
> The Office of Technology and Innovation spent nearly $600,000 to build out the foundations of the MyCity chatbot, which will be used for future chatbot offerings on MyCity. [0]

This was experimental tech... while I admire cities attempting to implement AI, it seems they did not spend enough tax dollars on it!

[0] https://abc7ny.com/post/ai-artificial-intelligence-eric-adam...

andsoitis 1 hour ago||
Why did NYC release it in the first place? Did they not QA it?

Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?

drillsteps5 25 minutes ago||
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking.
mulmen 6 minutes ago||
[delayed]
elgenie 55 minutes ago|||
QA efforts can whack-a-mole some issues, but the mismatch of problem and solution is inherent in any situation in which a generator of plausible-sounding text gets pointed at an area where correctness matters.
thedanbob 47 minutes ago|||
> Why did NYC release it in the first place? Did they not QA it?

Considering Louis Rossmann's videos on his adventures with NYC bureaucracy (e.g. [0]), the QAers might not have known the laws any better than the chat bot.

[0] https://www.youtube.com/watch?v=yi8_9WGk3Ok

fragmede 51 minutes ago|||
Why do you think OpenAI let a red team loose on GPT-5 for six months before releasing it to the public?
bluGill 1 minute ago||
For the image. There is no way a red team can find all the issues in 6 months. They can find some of the biggest, but even getting all the issues fixed in 6 months seems unlikely.
erxam 1 hour ago||
> Why did NYC release it in the first place?

Perhaps a big fat check was involved.

sylens 2 hours ago||
> The bot, built using Microsoft’s cloud computing platform

When is the last time there was positive news involving Microsoft? This bot could've easily been on AWS or GCP but I find it hilarious that here they are, getting dragged yet again

embedding-shape 1 hour ago||
https://iet.ucdavis.edu/content/microsoft-releases-xpsp2
walterbell 23 minutes ago|||
MS 2004
fragmede 52 minutes ago|||
golf clap
paxys 1 hour ago||
Even if the capability of each platform was exactly the same, Microsoft cloud users skew heavily towards governments, large non-tech corporations and really anyone who you sell to using large sales teams, fancy dinners and kickbacks rather than quality of software. And the end result follows.
kittikitti 27 minutes ago||
Being in and around the NYC area, while also knowing plenty of small businesses, I'm glad Mamdani killed this bot. Telling bosses to steal tips from their employees is run-of-the-mill corruption and common over here. The vibe for businesses is that everyone has to be exploiting someone else or have a schtick. If you were to talk about morals, you would be ridiculed. Most lawyers wouldn't even prosecute small businesses for this. It's probably why the agent was put into production, the level of business ethics in NYC is cartoonishly evil.
toomuchtodo 1 hour ago||
> A spokesperson for the mayor, Dora Pekec, confirmed in a text message that the new administration plans to take down the chatbot. She said a member of the Mamdani transition team had seen reporting on the bot from The Markup and THE CITY and presented it to the mayor as a possible place to save funds.

Journalism works.

atq2119 11 minutes ago|
It does. And it works best if you elect politicians who are willing to listen.
cmiles8 31 minutes ago||
We’ll likely see a lot of these AI pet projects get axed in the coming year or two… especially things rushed out in the early phases of the AI bubble when folks were desperate to appear to be using AI.
chasd00 5 minutes ago|
yeah i hope the problems stay to somewhat humorous themes like convincing a car sales bot to sell you a car for $1 and not more serious issues like convincing a bot to metaphorically launch the ICBMs.
terespuwash 34 minutes ago||
What else to expect from Eric Adams.
monero-xmr 54 minutes ago|
[flagged]
geoffeg 45 minutes ago|
To ride NYC's free busses, you must have a two minute conversation with a chat bot. (/s)