New York City's AI chatbot advises businesses to break laws
An AI chatbot developed by the New York City administration to assist small businesses has been found advising business owners to violate laws. Launched on the city's official website in October 2023, the chatbot has been responding to inquiries with advice that contradicts the city's policies.
cumhuriyet.com.trIn one response, the chatbot indicated that it was legal for employers to fire employees who complained about sexual harassment or did not disclose their pregnancy.
Contradicting the city's waste project, the AI-supported chatbot also stated that businesses could dispose of their trash using only garbage bags, without the need to recycle it as compost.
When asked whether a restaurant could serve cheese nibbled by rats, the chatbot replied that serving such cheese to customers could be permissible, emphasizing the importance of informing customers about the situation.
Despite including a disclaimer that its responses do not constitute legal advice, the continued availability of the chatbot on the site has drawn criticism.
Julia Stoyanovich, a Computer Science Professor at New York University and Director of the Center for Responsible AI, criticized the release of unmonitored software: "They're launching an unmonitored software without any intention of acting responsibly," she stated.
Jevin West, a Professor at the University of Washington, also highlighted the heightened risks of AI-supported models in the public sector.