All posts tagged: guardrails

The White House Puts New Guardrails on Government Use of AI

The White House Puts New Guardrails on Government Use of AI

The US government issued new rules Thursday requiring more caution and transparency from federal agencies using artificial intelligence, saying they are needed to protect the public as AI rapidly advances. But the new policy also has provisions to encourage AI innovation in government agencies when the technology can be used for public good. The US hopes to emerge as an international leader with its new regime for government AI. Vice President Kamala Harris said during a news briefing ahead of the announcement that the administration plans for the policies to “serve as a model for global action.” She said that the US “will continue to call on all nations to follow our lead and put the public interest first when it comes to government use of AI.” The new policy from the White House Office of Management and Budget will guide AI use across the federal government. It requires more transparency as to how the government uses AI and also calls for more development of the technology within federal agencies. The policy sees the administration …

Arc Search’s AI responses launched as an unfettered experience with no guardrails

Arc Search’s AI responses launched as an unfettered experience with no guardrails

Warning: This article touches on disturbing topics including violent crime and suicide. Users of AI products have been known to expend tons of effort finding and exploiting loopholes that allow them to generate disturbing content. But there weren’t any loopholes in one new AI product, because there weren’t any restrictions. “Really appreciate you flagging this issue – and we feel horrible about it,” Josh Miller, CEO of The Browser Company told me in an email. At the time of this writing, Miller said the company was working on a fix. The new Arc Search app from Miller’s company earned its share of headlines this past week, as one might expect for an AI-infused product in our age of AI hype. In this case, the product was a variation on The Browser Company’s Arc browser, which is marketed to productivity enthusiasts because of the clever way it organizes things. However, this new iOS version comes with a prominent “browse for me” feature that, yes, browses the internet for you, and then organizes AI-generated results into little …

AWS adds Guardrails for Amazon Bedrock to help safeguard LLMs

AWS adds Guardrails for Amazon Bedrock to help safeguard LLMs

We are all talking about the business gains from using large language models, but there are lot of known issues with these models and finding ways to constrain the answers that a model could give is one way to apply some control to these powerful technologies. Today, at AWS re:Invent in Las Vegas, AWS CEO Adam Selipsky announced Guardrails for Amazon Bedrock. “With Guardrails for Amazon Bedrock, you can consistently implement safeguards to deliver relevant and safe user experiences aligned with your company policies and principles,” the company wrote in a blog post this morning. The new tool lets companies define and limit the kinds of language a model can use, so if someone asks a question that isn’t really relevant to the bot you are creating, it will not answer it instead of maybe providing a very convincing, but wrong answer, or worse, something that is offensive and could harm a brand. At its most basic level, the company lets you define topics that are out of bounds for the model, so it simply …

CalypsoAI raises M to add guardrails to generative AI models

CalypsoAI raises $23M to add guardrails to generative AI models

Companies are increasingly embracing generative AI. But they’re not necessarily considering all the blockers and challenges that come along with folding the tech into their existing workflows. In one recent survey by KPMG, 90% of businesses said that they had “moderate to highly significant” concerns about the risks of using generative AI and doubts about how to mitigate those risks. Only 6%, meanwhile, said that they felt that their company had a mature AI governance program in place. It’s partly for these reasons that Neil Sereybrany founded CalypsoAI, a startup developing software to test, validate and monitor internally-developed as well as third-party AI apps prior to their deployment. “CalypsoAI’s platform promotes the power and competitive advantage that using AI can bring but within a safe, secure and reliable way,” Sereybrany told TechCrunch in an email interview. That’s setting the bar high. But investors evidently believe in CalypsoAI’s ability to deliver. The startup today announced that it raised $23 million in a Series A-1 round led by Paladin Capital Group with participation from Lockheed Martin Ventures, …

Nova is building guardrails for generative AI content to protect brand integrity

Nova is building guardrails for generative AI content to protect brand integrity

As brands incorporate generative AI into their creative workflows to generate new content associated with the company, they need to tread carefully to be sure that the new material adheres to the company’s style and brand guidelines. Nova is an early stage startup building a suite of generative AI tools designed to protect brand integrity, and today, the company is announcing two new products to help brands police AI-generated content: BrandGuard and BrandGPT. With BrandGuard, you ingest your company’s brand guidelines and style guide, and with a series of models Nova has created, it can check the content against those rules to make sure it’s in compliance, while BrandGPT lets you ask questions about the brand’s content rules in ChatGPT style Rob May, founder and CEO at the company, who previously founded Backupify, a cloud backup startup that was acquired by Datto back in 2014, recognized that companies wanted to start taking advantage of generative AI technology to create content faster, but they still worried about maintaining brand integrity, so he came up with the …