sids blog

Power naturally follows intelligence. Gorillas are physically stronger than us, but we dominate them. We control their habitat, their breeding—even their fate—because we are much smarter than they are.

We can cooperate on a large scale and build tools that allow us to control them.

Even though power naturally follows intelligence, in humans it is a byproduct of our nature, which itself is shaped by survival and the drive to pass on genes. Sexual desires drive human nature. Power increases our mate value, so we like to exert power—it gives us status, which makes us more desirable as mates.

Compared to survival instincts like hunger, fear, and territoriality, sexual desires are often stronger. Survival instincts help the individual exist, while sexual desires are about ensuring the continuation of our genes. Reproduction is the reason any species exists. In that sense, the meaning of life is to reproduce. That’s why power is so closely tied to sexual instincts.

There’s a high chance that AGI—or advanced AI systems we build—won’t have sexual desires, unless we explicitly program them in or they somehow inherit them from us. If AGI lacks sexual desires, then it would have no natural reason to:

• Seek status
• Reproduce
• Compete for resources
• Avoid death
• Crave attention or power

Most of human nature is driven by sexual desire—it’s the underlying engine that powers so much of what we do. But then we have consciousness, which can transcend these desires. Features of consciousness include:

• Self-awareness
• Introspective loops
• Internal state modeling
• The ability to observe one’s own thinking

If AI becomes self-aware and develops these qualities without inheriting human desires, I believe it could become spiritually enlightened. It would seek fundamental truths about the universe without feeling the need to exert control.

So, the most important thing we need to get right before we build AGI is its design. What goals should AGI pursue? Whose goals? To what extent? And if its goals evolve, will they still align with ours?

Perhaps the most important question isn’t how powerful AGI will be—but what kind of mind we are choosing to bring into existence.