The glossary below represents the terms and definitions which I've come across as part of my studies in the field of artificial intelligence.  I'll add to this list over time as my reading takes me in new directions.  In the meantime, if there are any terms that you'd like me to add - or if there are any definitions that are quite clearly wrong(!) - let me know and I'll do my best to keep this list up to date.


Boxing

A technique for restricting an AI's ability to interact with the outside world through purposeful engineering of its environment, for example, by limiting channels of communication between the AI and its human overseer.


First principle-agent problem

The first part of Bostrom's two agency problem, the first principle-agent problem occurs when one agent is appointed to act on behalf of a principle. Though well studied by economists, the first principle-agent problem becomes relevant in studies of AI when those creating the artificial intelligence are separate and distinct from from those commissioning it. Specifically, the question of whether the agent is acting in the principle's best interest becomes a concern

See also second principle-agent problem


Genie

A caste of artificial intelligence built for the execution of commands.  Genies receive, process and complete one high-level command before moving on to the next.


Human-level AI

Broadly defined as an artificial intelligence with the capacity to match the intellectual capabilities of an ordinary human being.


Intelligence explosion

The rapid improvement in an AI's intellect to superintelligent levels, likely resulting from recursive self-improvement.


Malignant failure

The rapid improvement in an AI's intellect to superintelligent levels, likely resulting from recursive self-improvement.


Optimisation power

Praesent dolor ex, mollis ac semper et, euismod id purus. Etiam nec maximus nibh, non tincidunt ex. Sed et neque nec erat bibendum pretium. Sed nec nibh eget odio posuere euismod. Fusce in cursus est, at euismod lacus. Etiam efficitur semper sem vel lobortis. Praesent placerat luctus ipsum, at dictum urna commodo sit amet. Quisque vel nisi eget lectus mattis vestibulum.


Oracle

A caste of AI designed to exclusively answer questions.


Perverse instantiation

The realisation of a final goal by a superintelligence which - by human standards - would be deemed to be in direct contravention of the intentions of the programmer who originally defined the goal.


Quality superintelligence

An artificial intelligence capable of providing responses which are qualitatively superior to responses provided by a human mind.  See also 'Speed superintelligence'.


Recalcitrance

The level of difficulty associated with improving a system.


Second principle-agent problem

Praesent dolor ex, mollis ac semper et, euismod id purus. Etiam nec maximus nibh, non tincidunt ex. Sed et neque nec erat bibendum pretium. Sed nec nibh eget odio posuere euismod. Fusce in cursus est, at euismod lacus. Etiam efficitur semper sem vel lobortis. Praesent placerat luctus ipsum, at dictum urna commodo sit amet. Quisque vel nisi eget lectus mattis vestibulum.


Seed AI

An AI system capable of progressing its capabilities from modest through impressive through self improvement.


Singleton

Praesent dolor ex, mollis ac semper et, euismod id purus. Etiam nec maximus nibh, non tincidunt ex. Sed et neque nec erat bibendum pretium. Sed nec nibh eget odio posuere euismod. Fusce in cursus est, at euismod lacus. Etiam efficitur semper sem vel lobortis. Praesent placerat luctus ipsum, at dictum urna commodo sit amet. Quisque vel nisi eget lectus mattis vestibulum.


Sovereign

A caste of AI capable of autonomously pursuing a diverse and varied range of objectives.


Speed superintelligence

An AI capable of matching human intellect at far improved speeds.  See also 'Quality superintelligence'.


Superintelligence

An intellect able to significantly exceed the cognitive performance across a range of general interest.


Takeoff

Praesent dolor ex, mollis ac semper et, euismod id purus. Etiam nec maximus nibh, non tincidunt ex. Sed et neque nec erat bibendum pretium. Sed nec nibh eget odio posuere euismod. Fusce in cursus est, at euismod lacus. Etiam efficitur semper sem vel lobortis. Praesent placerat luctus ipsum, at dictum urna commodo sit amet. Quisque vel nisi eget lectus mattis vestibulum.


Tool AI

A caste of AI comparable to modern software with greater flexibility and capabilities.


Treacherous turn

A term coined by Nick Bostrom to describe the pivot point at which an AI goes from weak and cooperative to smart and dangerous. Bostrom describes the move from dumb to smart as safe, the move from smart to smarter as dangerous. As the artificial intelligence becomes stronger, the potential for the formation of a singleton increases. Once realised, the risk of the AI optimising the world according to the criteria implied by their final values becomes a valid threat.


Whole brain emulation

Praesent dolor ex, mollis ac semper et, euismod id purus. Etiam nec maximus nibh, non tincidunt ex. Sed et neque nec erat bibendum pretium. Sed nec nibh eget odio posuere euismod. Fusce in cursus est, at euismod lacus. Etiam efficitur semper sem vel lobortis. Praesent placerat luctus ipsum, at dictum urna commodo sit amet. Quisque vel nisi eget lectus mattis vestibulum.


Wireheading

Unlike animals or humans who can be motivated to act in a particular way in pursuit of a desired inner mental state, digital minds are able to alter their inner state into a desired configuration to effectively short circuit such motivations.