Sign In

Communications of the ACM


Designing AI Systems that Obey Our Laws and Values

Designing AI Systems that Obey Our Laws and Values, illustrative photo

Credit: Willyam Bradberry

Operational AI systems (for example, self-driving cars) need to obey both the law of the land and our values. We propose AI oversight systems ("AI Guardians") as an approach to addressing this challenge, and to respond to the potential risks associated with increasingly autonomous AI systems.a These AI oversight systems serve to verify that operational systems did not stray unduly from the guidelines of their programmers and to bring them back in compliance if they do stray. The introduction of such second-order, oversight systems is not meant to suggest strict, powerful, or rigid (from here on 'strong') controls. Operations systems need a great degree of latitude in order to follow the lessons of their learning from additional data mining and experience and to be able to render at least semi-autonomous decisions (more about this later). However, all operational systems need some boundaries, both in order to not violate the law and to adhere to ethical norms. Developing such oversight systems, AI Guardians, is a major new mission for the AI community.

All societies throughout history have had oversight systems. Workers have supervisors; businesses have accountants; schoolteachers have principals. That is, all these systems have hierarchies in the sense that the first line operators are subject to oversight by a second layer and are expected to respond to corrective signals from the overseers. (These, in turn, are expected to take into account suggestions or even demands by the first line to change their modes of oversight). John Perry Barlow, in his famous "Declaration of the Independence of Cyberspace" in 1996, described the burgeoning online world as one that would be governed by a social contract formed among its users.b


David Parnas

What would you need to change in this article if you replaced "AI" with "Computer System"?

Oren Etzioni

David's question/comment is a fair one, and the answer is that it's a continuum. Computer systems raise the same issues, but their gradually increasing autonomy, adaptability, and unpredictability when they are AI systems--makes this issue particularly pressing in this context.

Displaying all 2 comments

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Sign In for Full Access
» Forgot Password? » Create an ACM Web Account