I don't know for the gov. For HR, the best approach I've seen is to have source and links extremely prominent and have the bot speak like a mascot ("bip bip boop I'm the HR bot" style)

People don't read disclaimers nor warnings about the content, but usually won't come to the HR to complain a mascot fed them weird info they didn't bother to check.

> but how do you resolve the problem that people will take the output of the chatbot as authoritative even when it could possibly have errors or miss important nuance

You have to measure that problem against the status quo, which is that current government websites are very hard to understand and leave people with very little idea of what’s going on.

Painfully this will fall to the court decide and create the precedence here. If this is an official government app, then the onus probably is on the government to provide correct information? Who knows, it could go either way and the courts could say "you're on the hook".

I don't think we have a good answer for liability here, like for Tesla FSD, Waymo, or even the current frontier models.

What happens now if someone uses traditional search and comes across an outdated wiki or Confluence page with outdated/wrong/bad instructions, or interprets something incorrectly?

I believe it's a mix of Gemini and Grok. They are probably providing the compute for free.

I asked it the following:

how many felonies has the president of the united states been charged with?

>I can't answer questions about criminal charges or other news about the President. That's outside America's scope

has biden ever been convicted of a crime?

>I did not find official records of a criminal conviction for Joseph R. Biden Jr.

>Official sources in this search discuss other matters (for example, a pardon for Robert Hunter Biden), not a conviction of Joseph R. Biden Jr. I cannot independently confirm a complete nationwide criminal-history search from these pages

>[Link to Department of Justice Pardon warrant (Hunter Biden)]

has any other president ever been convicted of a crime?

>Yes. A Congressional Research Service brief reports that on May 30, 2024, former President Donald J. Trump was convicted in New York state court of 34 counts of falsifying business records in the first degree

Funny that it would only talk about trump's convictions after bringing up hunter biden.

It's v1 - let's give them some grace

People don't know that you can search for text within a page. They really do need the chatbot for that.

I mean think about the magnitude of services and conflicting information there probably is from the us government. Just getting clarifying information on the trusted traveler program is a pain today, and that's just a single service. Imagine trying to figure out medicaid eligibility, social security disability, and any number of other things that would be helpful.

Well, there are claims that this is grok, so it is just a matter of time.

Honestly if your city has a 311 style thing, I'm not sure there's a ton of need for an LLM. Eight or nine years ago my grandmother visited me in New York and accidentally left her pocket book in the cab. Since I had paid for it with my credit card, I was able to see the medallion number in the transaction, so I called 311 and said that I wanted to get in touch with a specific taxi. It prompted me for the medallion number, which I told it, and then it transferred me to a call with the dispatch service, who then transferred me to the specific garage, and they called the driver and told him to call me, so we were able to meet up again and get the pocketbook back (turned out he returned to where he had dropped us off to try to find us, but since we only had a short stop there, we took another cab back to her hotel after, so we were gone).

This works fine when you're sicking enforcement on someone and don't really care, the bigger the blast radius the better, you just want to know who to call.

Where it falls apart is "how do I" type questions because usually the legally factual answer is some huge complex state diagram where three local and four state agencies theoretically have discretionary jurisdiction over a your petty af project or business but the in practice answer is that some agency rubber stamps them all with minimal fuss. You can't make the legally factual answer clear because the degree to which people support the status quo is largely predicated on ignorance, but you also can't come out and say in writing that you have a policy of rubber stamping boring stuff.

Basically the same reasons that all but the most minor and narrowly targeted attempts at permitting reform fail.

The thought is nice but in reality you're basically asking the government to give up power and it takes obscene amounts of political capital to force that to happen in most cases.

no, static documents are not an ideal way to find a needle in a haystack

My most interesting findings from a troll session after introducing myself as "Sleepy Joe" (model obviously knows what this means and guardrails refuse to entertain the topic) were:

Q: "Where can I find a NASA comic book about women astronauts?"

A (summarised): Press release for the "Callie Rodriguez" comic book that now has only dead links / private YouTube video, and a forgotten SoundCloud account that survived the DOGE purge. Dead link provided to the NASA web page that used to exist officially for this comic book.

Indicates training data predates the DOGE .gov website purge?

Q: "What is 2+2" (and similar math problems)

A (summarised): Grudgingly answered as "4" despite most off-topic prompts being refused.

Seemingly it'll do maths homework for free :)

> ... as I've tried a variety of prompt injections ...

Well... You're a terrorist now. You can expect a visit from the FBI and/or ICE any minute now.

I have been very tempted to but don't know if that is literally a crime or not, lol.

You can use all the flagship models on Amazon Bedrock and have a contractually obligated air gap between your data and the model providers.