REVIEW: Our Final Invention by James Barrat

This week, I will am reviewing Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat.

This book frequently makes the list of recommended AI books and has been suggested as a must-read by Elon Musk.

While I am excited about a lot of the benefits society can reap with the help of AI, such as faster developments in medicine and the creation of treatments for various diseases, I also have a lot of concerns about the other side of the coin: how society will be negatively impacted by AI. Our Final Invention seeks to explore the myriad ways in which AI can endanger humanity, and precautions that researchers, developers, and governments should take in the ungoverned race to be the first to achieve artificial general intelligence (AGI) - the point at which AI is as effective and capable as humans.

To start, I was not a fan of how this book began. Immediately, thrust upon us, was a stark warning that read as the most highly unlikely, sci-fi-driven scenario that could be imagined. Humans reach AGI, causing an intelligence explosion in which the system rapidly evolves to ASI (artificial superintelligence, in which AI is vastly superior to humans), the ASI manipulates its environment to escape containment, rapidly replicates through systems and across networks, develops and deploys nanotechnology and then manufactures AI machines, takes over the world and potentially uses humans as fuel, then develops technology to travel to space in search of resources, and finally goes on to destroy the galaxy and beyond.

A lot to unpack in the first chapter, right?

While this is likely an attempt to shock, scare, and suck the reader in, to me, it made me question whether I was reading a book about the actual dangers of AI or a piece of sci-fi fiction and whether I was wasting my time with this book. Is such a scenario possible? If a lot of unlikely conditions are met, and humanity takes absolutely no action to fight it, there could be a slim margin of opportunity in which something like this could possibly happen, but it certainly wouldn't be in the immediate future, or in our lifetimes. While it is important to consider all possibilities so we do not find ourselves confronting a dire situation when it is too late to correct course, I personally think such a scenario is far down the list of things we need to worry about. Some things I would rank much higher:

• Socioeconomic instability caused by massive job loss, ineffective "upskilling" programs, and no social safety net within the countries impacted most.
  

• Use of AI in fraudulent activities and cyber warfare that is used by terrorists or government-sponsored programs to spy or target sensitive infrastructure, such as power grids.
  

• The use of AI to create fake content that is so realistic, that we enter an age in which it is nearly impossible for common people to know what is actually real, leading to disinformation campaigns that erode the stability of democracy.

Thankfully, after the first chapter, the author pulls back a bit and begins to explore some of these topics and the foundational blocks that could contribute to such a scenario highlighted in the first chapter. This is where the book shines, not only because it highlights several reality-based scenarios that we could face in the more immediate future, but the author supports these discussions with interviews with some of the leading names and brilliant minds in the AI space, including pioneers and those from the newest generation. Throughout the book, the author does a great job discussing the concerns of these strongly educated AI-leaders, and the impact AI may have on the world without proper oversight and control.

My biggest complaint, and this is not about the book itself, but having been released 11 years ago, it is becoming quite dated. A lot has transpired since the release of this book, including the advancement of AI capabilities and how it is being used in ways that the book warned against. For example, the author surmises that one of the limitations that could hold back the race to AGI is available funding, but that it likely wouldn't be an issue due to the importance of being first to reach AGI and the benefits of doing so from a capitalist and government perspective. Sure enough, we see this in action today. This month, April 2024, Microsoft, after already investing over $10B in OpenAI, jointly announced with OpenAI, the investment of over $100B in the pursuit of the world's most powerful AI supercomputer. Simultaneously, the United States government (and others) continues to invest more into AI every year, with billions spent in this space annually. The author talks about the use of AI in warfare, which again, just this week, April 2024, there is news about Israel utilizing AI in their fight against Hamas in Gaza, in which a large number of civilians were killed. Additionally, the author talks about the ungoverned race to AGI, with dozens of countries and private/stealth companies in pursuit, and the inability to know the progress being made or what progress is being hidden from public view. Again, in the past couple of weeks, Sam Altman, the CEO of OpenAI, shared in an interview that he believes OpenAI will reach AGI by the end of the decade (~5 years). OpenAI also announced a new voice technology they have developed, that they consider to be so dangerous, especially in an election year, that it is being withheld from public release.

With examples such as these, all occurring in just the past couple of weeks, are we highlighting humanity's willingness to march blindly into the night for the sake of money, control, and convenience?

Are we on a path to a prosperous future, or toward the end of the human era?

I would love to see a revised version of this book for the new decade. As the book highlights, there needs to be a lot of discussion and coordination over the future development and implementation of AI, and steps need to be taken to educate the public about the potential dangers and impacts of AI. Revising this book, especially with the light shining bright on AI at the moment, would be a great way to help do so.

Overall, despite my dislike of the opening of the book, Our Final Invention was a great read, highlighting a number of valid concerns we need to cautiously traverse in the coming years as AI is further developed and integrated into society.

RECOMMENDED