PapeRman #6

本文描述了一个新的推断智能体动机的方法。该方法基于影响图,这是一种图模型的类型,包含特别的决策和效用节点。图标准可以被用来确智能体观测动机和智能体干预动机**

Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings

Tom Everitt Pedro A. Ortega Elizabeth Barnes Shane Legg
Deepmind

Agents are systems that optimize an objective function in an environment. Together, the goal and the environment induce secondary objectives, incentives. Modeling the agent-environment interaction in graphical models called influence diagrams, we can answer two fundamental questions about an agent’s incentives directly from the graph: (1) which nodes is the agent incentivized to observe, and (2) which nodes is the agent incentivized to influence? The answers tell us which information and influence points need extra protection. For example, we may want a classifier for job applications to not use the ethnicity of the candidate, and a reinforcement learning agent not to take direct control of its reward mechanism. Different algorithms and training paradigms can lead to different influence diagrams, so our method can be used to identify algorithms with problematic incentives and help in designing algorithms with better incentives.

This image has an empty alt attribute; its file name is screenshot-from-2019-01-17-01-49-57.png

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s