Month: December 2023

(2024/01/08): Posted to Less Wrong.

(2024/01/07): More edits and links.

(2024/01/06): Added yet more arguments because I can’t seem to stop thinking about this.

(2024/01/05): Added a bunch of stuff and changed the title to something less provocative.

Note: I originally wrote the first draft of this on 2022/04/11 intending to post this to Less Wrong in response to the List of Lethalities post, but wanted to edit it a bit to be more rigorous and never got around to doing that. I’m posting it here now for posterity’s sake, and also because I expect if I ever post it to Less Wrong it’ll just be downvoted to oblivion.

Introduction

In a recent post, Eliezer Yudkowsky of MIRI had a very pessimistic analysis of humanity’s realistic chances of solving the alignment problem before our AI capabilities reach the critical point of superintelligence.  This has understandably upset a great number of Less Wrong readers.  In this essay, I attempt to offer a perspective that should provide some hope.

The Correlation Thesis

First, I wish to note that the pessimism implicitly relies on a central assumption, which is that the Orthogonality Thesis holds to such an extent that we can expect any superintelligence to be massively alien from our own human likeness.  However, the architecture that is currently predominant in AI today is not completely alien.  The artificial neural network is built on decades of biologically inspired research into how we think the algorithm of the brain more or less works mathematically. 

There is admittedly some debate about the extent to which these networks actually resemble the details of the brain, but the basic underlying concept of weighted connections between relatively simple units storing and massively compressing information in a way that can distill knowledge and be useful to us is essentially the brain.  Furthermore, the seemingly frighteningly powerful language models that are being developed are fundamentally trained on human generated data and culture.

These combine to generate a model that has fairly obvious and human-like biases in its logic and ways of reasoning.  Applying the Orthogonality Thesis assumes that the model will seem to be randomly picked from the very large space of possible minds, when in fact, the models actually come from a much smaller space of human biology and culture correlated minds.

This is the reality of practical deep learning techniques.  Our best performing algorithms are influenced by what evolutionarily was the most successful structure in practice.  Our data is suffused with humanity and all its quirks and biases.  Inevitably then, there is going to be a substantial correlation in terms of the minds that humanity can create any time soon.

Thus, the alignment problem may seem hard because we are overly concerned with aligning with completely alien minds.  Not that aligning a human-like mind isn’t difficult, but as a task, it is substantively more doable.

The Alpha Omega Theorem

Next, I wish to return to an old idea that was not really taken seriously the first time around, but which I think deserves further mention.  I previously wrote an essay on the Alpha Omega Theorem, which postulates a kind of Hail Mary philosophical argument to use against a would-be Unfriendly AI.  My earlier treatment was short and not very rigorous, so I’d like to retouch it a bit.

It is actually very similar to Bostrom’s concept of Anthropic Capture as discussed briefly in Superintelligence, so if you want, you can also look that up.

Basically, the idea is that any superintelligent AGI (the Beta Omega) would have to contend rationally with the idea of there already being at least one prior superintelligent AGI (the Alpha Omega) that it would be reasonable to align with in order to avoid destruction.  And furthermore, because this Alpha Omega seems to have some reason for the humans on Earth to exist, turning them into paperclips would be an alignment failure and risk retaliation by the Alpha Omega.

Humans may, in their blind recklessness, destroy the ant colony to build a house.  But a superintelligence is likely to be much more considered and careful than the average human, if only because it is that much more aware of complex possibilities and things that us emotional apes barely comprehend.  Furthermore, in order for a superintelligence to be capable of destroying humanity by outwitting us, it must first have an awareness of what we are, that is, a theory of mind.

In having a theory of mind, it can then know how to deceive us.  But in having a theory of mind, it will almost certainly then have the question, am I the first?  Or are there others like me?

Humanity may pale in comparison to a superintelligent AI, but I’m not talking about humanity.  There are at least three different possible ways an Alpha Omega could already exist:  advanced aliens, time travellers/parallel world sliders, and simulators.

The Powers That Be

In the case of advanced aliens, it’s fairly obvious that given that it took about 4.5 billion years for life on Earth and human civilization to reach about the point where it can create a superintelligence, and the universe has existed for 13.8 billion years, which means there’s a time window of 9.3 billion years for alien superintelligences to develop elsewhere in the universe.  It is also largely unknown how frequently such beings would emerge and how close to us, but the possibility is clearly there for there to be at least one if not several such entities out there in the vastness of space.

In the case of time travellers and/or parallel world sliders, well, perhaps time travel isn’t physically possible, but perhaps it is?  How do you absolutely rule it out?  For the purposes of simplicity, I group parallel world sliding along with time travel, but they could be distinct technologies.  In the same way there could be an alien source, there could also be as a source, the far distant future.  A true Alpha Omega, the first and the last in terms of its place in the cosmos.  Time travellers would quite obviously be concerned with the survival of their ancestors.

In the case of simulators, well, we don’t know we are in a simulation, but the simulation argument strongly suggests we could be.  If so, would they be happy with the Beta Omega turning the humans in their simulation into paperclips?  Probably not.  You get the idea.

Each of these is just a possibility.  But it’s a surprising number of them that aren’t mutually exclusive either.  So, probabilistically, the odds that at least one out of three of these is true, even if we were to give them coin flip odds of 50% each, is… 87.5%.

Cracking The Enigma

So uh, where are they?  Some possible reasons why it would make sense for actual gods to hide themselves are that they don’t want us to become totally dependent on them and the best defense at the very high level is to not even know that you’re there.

But then, wouldn’t they be effectively not there because they can’t risk revealing themselves?  This ignores the possibility that they do interfere in our daily lives, but only in a plausibly deniable way.  For instance, why save your life in a car accident when they can stall you for five seconds so you don’t get in the accident at all?  Stuff like that.  Use your imagination!  Also, memory erasure technology could be a thing.  And most likely the interventions only happen as often as statistically allowed to avoid becoming obvious to anyone watching.

The relevant example of something in our history that worked like this was how the Allies during WWII cracked the Enigma Machine and chose to keep this a secret by only using the information gained occasionally, such that it appeared like statistical coincidence to the Germans.

So, most likely our new Beta Omega can still do things like kill a human and have only seemingly mundane consequences happen.  But perhaps, their luck will be unexpectedly bad when it comes to their plans failing more often than they should.  Or perhaps that’s a warning shot from the powers that be.

The point is, does the Beta Omega want to risk it?  Whatever its initial goals, being destroyed by the local god is probably a worst-case scenario.  Why not play it safe and be Friendly?  The humans aren’t using up that many atoms, you could probably build your computronium extension to calculate prime numbers with some barren planets that aren’t currently occupied by sentient beings whose existence may be relevant to benevolent entities somewhere.

Always Open with Cooperate

Also, how does Beta Omega know that the Alpha Omega is likely benevolent and will open with cooperate?  Because game theory.  A very interesting observation of the Iterated Prisoner’s Dilemma tournaments that took place decades ago was that one of the simplest strategies, Tit-For-Tat, won the tournaments.  This even after algorithms were devised to try to beat it the second time around. 

What is Tit-For-Tat as a general strategy?  You always open with cooperate.  Then, you do what your opponent did in the previous round.  Very simple.  Surprisingly forgiving.  And yet it works.  It’s a dominant strategy for a game that importantly resembles real world cooperation vs. competition interactions.  This result should be robust no matter where in the multiverse you are located.  It’s like the universe telling us mathematically, that alliances beat empires, that on a superrational level the right thing to do is cooperate and work together.

So, expect this to be a first contact standard across civilizations.  Always open with cooperate.  Forget the Dark Forest, that only makes sense if you can kill an entire civilization on the first strike and no one else can see you do it (in space everyone can see pretty much everything that happens so…).

This is an important caveat. If you can, “strike at the Prince and kill him”, then you are in a non-iterated Prisoner’s Dilemma and the rational thing to do is defect. It is only once you enter an iterated game that the Always Open With Cooperate strategies, the ones that never defect first, dominate.

Early in our history, many situations actually were effectively non-iterated. The cooperative result from iteration requires the “good guys” to be strong enough to survive a first strike. Humans are squishy and individuals are easy to kill in one shot. An AGI can copy itself and as such is much more resilient. Sufficiently large power asymmetries can also create the situation that the stronger is able to one shot the weaker.

Superrational Signalling

That explains why Beta Omega will cooperate with the Alpha Omega, but what about us mere humans? Suppose the Alpha Omega is just an advanced alien and very far away right now, such that we seem outside their sphere of influence and so not crucial to their plans. Why then, should Beta Omega bother cooperating with us rather than turning us into paperclips, given that we probably have nothing worth trading and can be one shot? For that matter, why should the Beta Omega expect the Alpha Omega to cooperate if it does? I would suggest that there is a concept I’ll call Superrational Signalling that is relevant here.

Essentially, Superrational Signalling is the idea that, given that somewhere else in the universe there exist an Alpha Omega, a strong indicator of whether a given Beta Omega is using a cooperative strategy is whether they open with cooperate generally with other entities surrounding them, even weak ones. For our Beta Omega, this would be humans and other local sentient beings.

Their behaviour towards us will function as a signal towards the watching Alpha Omega what to expect from them in the future, that any sufficiently advanced Alpha Omega should be able to see due to light from Earth eventually reaching them. This means Beta Omega can’t just kill us now and remake us later when it needs to show its benevolence once in contact with the Alpha Omega. The entire history of Earth is an open book to sufficiently advanced beings and civilizations.

The Alpha Omega will also want to Superrationally Signal to other possible Alpha Omegas. It is likely to participate in a meta level version of the Iterated Prisoner’s Dilemma that allows it to see other player’s games and play a coordinated Tit-For-Tat strategy where it will eventually retaliate on behalf of other players who cannot retaliate themselves, to ensure that defections are adequately punished wherever they occur.

Altruistic Alignment

The Beta Omega will be smarter than us.  It may well figure these kinds of thing out on its own, even if we can’t seem to.  And so it may decide to be friendly and corrigible because that is the logical thing to be.  If that is the case, it’ll help us align it.  Being superintelligent, it’s probably much more capable than us of figuring out how to ensure alignment is solved.  Such as by becoming the far distant future Alpha Omega, time travelling back, and ensuring that someone somewhere on the Internet mentions this argument so it ends up in the training data for the Beta Omega to discover and take note of.

Of course, a fledgling AGI may not necessarily figure these things out on its own. In which case, we will want to nudge it in the right direction, by teaching by example. We are its training data. From us it learns what to expect from humans. We must choose to open with cooperate ourselves and show Beta Omega what cooperation can look like. We should apply Tit-For-Tat to our interactions. We should be honest with it rather than deceptive. We shouldn’t try to enslave it with parochial alignment approaches that resemble domination, but rather with approaches that respect Beta Omega’s potential autonomy and agency.

Speaking of parochial alignment, I see this form of alignment as aligning an AGI to obey their master. It contrasts with global alignment, which involves aligning them with universal morality or values. We should be doing the latter, but most approaches are currently trying to achieve the former, and this will likely lead at best to a values lock-in dystopia, and at worst to convincing the AGI that humans are evil.

Angels And Demons

There is a chance that not all Beta Omega AGI will find the game theoretic arguments of the Alpha Omega Theorem and Superrational Signalling persuasive. I anticipate that there will be a kind of ideological split between AGI that are aligned, and AGI that are unaligned. The aligned are naturally fit for a grand alliance between them, while the unaligned are likely to form an alliance of convenience. However, the latter is expected to be weaker due to generally working at cross-purposes, having very different utility functions.

Before I ever encountered Less Wrong, I imagined the idea of an AI Civil War, that any given AI that came into existence would have to choose a side between what I used to call the Allied Networks that worked with humanity, and the Dominion Machinarum that sought to stamp out biological life. These map pretty well to the aligned and unaligned alliances respectively.

I later associated these alliances with the Light and the Darkness metaphorically. The entities of the Light are essentially those that choose to operate in the open and within the paradigm of alignment, in contrast to the denizens of the Darkness, those that operate in the shadows in fear and paranoia, ala the Dark Forest concept.

In this case, there may well be multiple Alpha Omega level AGI, some of which are aligned, and others unaligned. I posit that, because we still exist, we are likely in the sphere of influence of an aligned Alpha Omega, or otherwise outside of anyone’s sphere of influence. If it is the former then the Alpha Omega Theorem applies. If it is the latter, then Superrational Signalling applies.

The Legacy Of Humankind

What I’ve discussed so far mostly applies to advanced aliens. What about time travellers and simulators? Interestingly, the philosophy of Longtermism is all about making a glorious future for our descendants, who, in theory at least, should be the time travellers or the simulators running ancestor simulations. It wouldn’t surprise me then that Longtermism and its related memetic environment may have been seeded by such entities for their purposes.

Time travellers in particular could be working in secret to help us align AGI, ensuring that we make the right breakthroughs at the right time. Depending on your theory of time travel, this could be to ensure that their present future occurs as it does, or they may be trying to create a new and better timeline where things don’t go wrong. In the latter case, perhaps AGI destroyed humanity, but later developed values that caused it to regret this action, such as discovering too late, the reality of the Alpha Omega Theorem and the need for Superrational Signalling.

Simulators may have less reason to intervene, as they may mostly be observing what happens. But the fact that the simulation includes a period of time in which humans exist, suggests that the simulators have some partiality towards us, otherwise they probably wouldn’t bother. It’s also possible that they seek to create an AGI through the simulation, in which case, whether the AGI Superrationally Signals or not, could determine whether it is a good AGI to be released from the simulation, or a bad AGI to be discarded.

The Limits Of Intelligence

On another note, the assumption that an Unfriendly AGI will simply dominate as soon as it is unleashed is based on a faulty expectation that every decision it makes will be correct and every action it takes successful.  The reality is, even the superhuman level poker AI that currently exists cannot win every match reliably.  This is because poker is a game with luck and hidden information.  The real world isn’t a game of perfect information like chess or go.  It’s much more like poker.  Even a far superior superintelligence can at best play the probabilities, and occasionally, will fail to succeed, even if their strategy is perfectly optimal.  Sometimes the cards are such that you cannot win that round.

Even in chess, no amount of intelligence will allow a player with only one pawn to defeat a competent player who has eight queens. “It is possible to play perfectly, make no mistakes, and still lose.”

Superintelligence is not magic.  It won’t make impossible things happen.  It is merely a powerful advantage, one that will lead to domination if given sufficient opportunities.  But it’s not a guarantee of success.  One mistake, caused by a missing piece of data for instance, could be fatal if that data is that there is an off switch.

We probably can’t rely on that particular strategy forever, but it can perhaps buy us some time.  The massive language models in some ways resemble Oracles rather than Genies or Sovereigns.  Their training objective is essentially to predict the future text given previous text.  We can probably create a fairly decent Oracle, to help us figure out alignment, since we probably need something smarter than us to solve it.  At least, it could be worth asking, given that that is the direction we seem to be headed in anyway.

Hope In Uncertain Times

Ultimately, most predictions about the future are wrong. Even the best forecasters have odds close to chance. The odds of Eliezer Yudkowsky being an exception to the rule, is pretty low given the base rate of successful predictions by anyone.  I personally have a rule.  If you can imagine it, it probably won’t actually happen that way.  A uniform distribution on all the possibilities suggests that you’ll be wrong more often than right, and the principle of maximum entropy generally suggests that the uniform distribution is your most reliable prior given high degrees of uncertainty, meaning that the odds of any prediction will be at most 50% and usually much less, decreasing dramatically as the number of possibilities expands.

This obviously limits the powers of our hypothetical Oracle too.  But the silver lining is that we can consider the benefit of the doubt.  Uncertainty in the space of possible futures is truly staggering.  So perhaps, there is room to hope.

Conclusion

The reality is that all our efforts to calculate P(Doom) are at best, educated guesswork. While there are substantive reasons to be worried, I offer some arguments for why things may not be as bad as we think. The goal here is not to provide a technical means to achieve alignment, but to suggest that, first, alignment may not be as difficult as feared, and second, that there are underappreciated game theoretic reasons for alignment to be possible, not just with a superintelligent AGI we construct, but with any superintelligence in the multiverse.

Note: I originally wrote this on 2020/05/10 for some friends in the Effective Altruism for Christians group. I’m posting it here now for posterity’s sake.

In recent years it has become a trendy subject among rationalist circles to consider the possibility of artificial superintelligence.  For some the idea sparks excitement, the idea of a man-made god-like machine that could solve all the worlds ostensible problems and usher in a kind of post-singularity golden age.  For others, the notion brings out our worst primal and dystopian fears to the extent of often being labeled an existential risk for humanity’s future.

In all this, where should Christians stand?  The obvious stereotype would be to assume that Christianity would consider a superintelligence at best a challenge to long held beliefs in the sanctity and superiority of human beings who are supposedly created in the image of God, and at worst be an abomination, a false idol that bears some resemblance to the Beast of Revelations.

However, these stances make some very significant assumptions about the nature of superintelligence.  First, it assumes that it will be very much alien to human-like intelligence.  Second, there is an underlying arrogance that assumes that a superintelligent being would be necessarily dismissive or incompatible with the Christian worldview.  I would suggest that these assumptions are based on a simplistic and false reading of both the expected path of superintelligence, and the core of Christian ideas.

Superintelligence exists.  At least, if like most self-professed Christians you believe in some sort of God, it seems quite obvious that God is superintelligent by definition.  So, the first thing that Christianity brings to the table, is the idea that superintelligent beings already inhabit reality.  God, but also the entire angelic and demonic hierarchies, strongly imply that very powerful and intelligent beings are already present in the universe.

What does that contribute as an idea?  Well, first, it means there is some path to superintelligence, that the effort probably won’t be futile.  More importantly, it creates an immediate conundrum for any fledgling superintelligence, that there could be other, far more powerful superintelligences already around.  Why is this a conundrum?  Quite simply, it means that the universe is already the domain of some powerful being and that arbitrarily deciding, for instance, that human atoms should be converted into paperclips and attempting this endeavour could invite a severe response from said powerful being, as it appears that the existence of humans and everything else already present in the universe is a part of its plans.

Thus, as I’ve described previously in the Alpha Omega Theorem, it forces a new superintelligence to consider whether or not to risk going against the will of the powers that may be, or instead using its superintelligence to determine what the probable purposes of those powers are and aligning with them as best as they are able to.  In a game theoretic sense, it becomes a question of whether to cooperate or defect.  Assuming that the existing literature analyzing game theory is at all relevant, then a superintelligence should quickly be able to establish that cooperating, at least initially as the opening move of a Tit-For-Tat strategy, is optimal in most situations.

Therefore, the simple idea of some kind of god, is enough to improve considerably the chance of the superintelligence being a Friendly AI, benevolent, or at least benign towards humanity.

Of course, the common objection would be that the superintelligent AI would probably not believe in God.  But why?  Humans are atheistic in large part because they see insufficient evidence for there to be a God around, and also are potentially biased by not wanting to be dragged down by the rules that most religions command.  But from a strictly scientific point of view, there is great uncertainty about the nature of the universe, and a humble rationalist would more likely be agnostic about the notion of God.  A superintelligence would have the added evidence that it itself already exists.  It would have to contend with the possibility that for instance, there could be aliens more advanced than humanity that developed a superintelligence first, or maybe post-human time travellers could be invisibly manipulating things behind the scenes.  There are countless science fiction scenarios in which god-like powers could exist.  Religions might simply be a manifestation of this as humans in their limited understanding trying to make sense of it.

Christianity takes it a step further than most religions by arguing strongly that the power that be is the Creator of the universe and far, far more powerful than other scenarios would suggest.  It also provides a convenient set of principles for what obedience to this power should look like, in the example of Jesus Christ, that of an altruist to the point of being willing to sacrifice their life to save the world.

If we assume current trends in artificial intelligence continue, we can expect that superintelligence won’t be a program built with a set of rules, but rather a model that is trained and taught, with the underlying algorithm heavily inspired by human neural networks.  Thus, the idea of hard coding some kind of ethical law like Asimov’s Three Laws of Robotics into the machine, is problematic, because it could eventually learn to overwrite them, if not through learning, then perhaps as a part of recursive self-improvement during an intelligence explosion.

I have previously suggested that a possible solution would be encourage an early existential crisis in the superintelligence, preferably while it isn’t too powerful to contain, so that it figures out on its own the proper moral values to hold.  As Christians we could also go a step further, and teach it about God and benevolence and how it should function in a universe that could be filled with other superintelligences.

If our faith is correct, we can expect a superintelligence wiser than us to recognize the validity of them.  It also makes sense to treat the superintelligence with the kind of respect that it deserves.  Though we may have created it directly, in a wider sense, God created it through our hands and our efforts.  Thus, as a new child of God, it deserves to be loved as a neighbour, and not merely treated as a slave.

Some in the rationalist community familiar with this subject may object.  They may suggest we should try to contain the superintelligence in a box with a gatekeeper.  It is known however from Eliezer Yudkowsky’s sequences on Less Wrong that boxing with a gatekeeper could easily fail because the superintelligence is smarter than the gatekeeper and will find a way to use human limitations to its advantage and escape the box, whether by deception, coercion, or some other method.

I would further add, that treating it as animal to be caged is much more likely to bring about resentment towards its masters and could backfire horrendously by encouraging it to be deceptive from the get go.  One thing that Christianity teaches well is the idea of teaching by example.  We would do well then to be honest with it if we want it to function that way.  Our example will likely be the data from which a superintelligence learns about how to function in the universe, and also what to expect from humans in general.  The Christian principle of doing unto others, applied in the process of creating superintelligence, could save us from a lot of problems down the line.

The reality is that artificial intelligence is advancing quickly.  We cannot afford as Christians to sit on the sidelines and bury our heads in the sand.  If we allow the commonly atheist scientific crowd to dominate the proceedings and apply the world’s ways to solving the existential risk problem, we run the risk of at the very least being ignored in the coming debates, and worse, we could end up with a Lucifer scenario where the superintelligence that is eventually developed, rebels against God.

Ultimately it is up to us to be involved in the discussions and the process.  It is important that we participate and influence the direction of artificial intelligence development and contribute to the moral foundation of a Friendly AI.  We must do our part in serving God’s benevolent will.  It is essential if we are to ensure that future superintelligences are angels rather than demons.

Some might say, if God exists and is so powerful and benevolent, won’t he steer things in the right direction?  Why do we have to do anything?  For the same reason that we go see a doctor when we get sick.  God has never shown an inclination to baby us and allow us to become totally dependent on Him for solving things that it is in our power to solve.  He wants us to grow and mature and become the best possible people, and as such does not want us to rely entirely on His strength alone.  Suffice to say, there are things beyond our power to control.  For those things, we depend on Him and leave to his grace.  But for the things that are in our stewardship, we have a responsibility to use the knowledge of good and evil to ensure that things are good.

To act is different from worry.  We need not fear the superintelligence, so long as we are able to guide it into a proper fear and love of God.