Not only builders, the greatest takeaway for me is that everyone has a responsibility in shaping the discourse, culture, and usage of transformative technology. This "the builders will do the right thing" mentality is even (in my interpretation) explicitly called out in several places:
> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...
> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.
> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
Not new for software and hardware industry though, practitioners have just chosen to ignore it. From the Association for Computing Machinery, which encompasses all forms of software development, the very first principle is the public good:
"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:
1.01. Accept full responsibility for their own work.
1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.
1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."
From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:
"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;
2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."
hard disagree. No matter what gets built, it will be used for bad things if the society is not properly constructed. We need to fix our government and then you can build anything you want. I don't like putting the blame on the builders, building is already difficult as it is. Also you can't built something in isolation, word will get out anyway. That's why this speech is either naive or he just wanted to tick a todo list: say something about AI.
From a high-level view, there are powerful cultural/political forces that nudge us towards building harmful, wasteful things without us being aware of said forces. The government, corporate greed, and billion-dollar marketing budgets are to blame.
But on the ground, from an in-the-trenches perspective, I think an engineer at Meta, for example, shares in some level of blame, too. Mainly because there is plenty of evidence showing how much harm Meta products have brought upon society. Yet Meta couldn’t be built without engineers opting in to work there in exchange for $300k salaries.
We desperately need moral clarity and courage at both the policy-level and the individual level.
I get where you are coming from but this is the common "reduction to politics" that anyone who doesn't want to address a problem uses: think of any societal or human problem and you can have your comment with different nouns.
Sure, IF we could just go and fix our governments in some magical way then the problem would disappear. That goes from hunger, climate change, videogame addiction and AI. The problem is that what you value in life in different than what others do, so we now have a system in which sometimes you get what you want and sometimes you don't.
But back to the topic, I do think that how OpenAI and Anthropic handled the government and them asking to drop guardrails is something a company can actually and actively do without having to reinvent the universe.
Like it or not if you knowingly build stuff that is used for evil purposes you are complicit
You can't build an orphan mulching machine and get away with just a shrug. "I don't really agree with mulching orphans but someone else paid me to build that. If I didn't build it someone else would have" just does not absolve you of your involvement in mulching orphans
When he speaks of the builders needing to take responsibility, I think he is directing his comments at tech company leadership. The Sam Altmans of the world, not the poor slobs like us who need a job for health insurance because apparently unemployed people don't deserve to live.
No, he's talking to you too. The Sam Altman's can't be a threat without useful idiots to cooperate with and doing work for them. You don't get out of your own culpability because you locked on a set of golden handcuffs.
> Therefore builders "bear a particular ethical and spiritual responsibility"
> This is a message we need right now.
Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".
Yes but... No one living today knows what direction AI technology will take humanity. If we have an algorithm breakthrough then we may avoid building new data centers. If the abilities of the technology plateau then there might not be large impacts on employment. Builders need to focus on the impact of their next steps. Don't put polluting natural gas generators in neighborhoods today. Don't make unemploying folks the goal of your tool, today. Don't make decisions that harm people and the environment today.
> The Stratos artificial intelligence datacenter footprint will cover more than 40,000 acres (62 sq miles) over three sites in Box Elder county in north-western Utah. The facility will require about 9GW of power, which is more than the entire state of Utah currently consumes
Sure hope your rosy inflection point happens real soon.
Otherwise, the direction this is taking us is pretty obvious.
> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.
The most famous example may be the perpetual war between architects and engineers/builders.
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.
Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.
> I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology)
I believe the Amish figured this out over a century ago.
The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.
This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:
- Fuck you, got mine
- If I don’t do it, someone else will
- Might makes right
- Greed is good
It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.
Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap
- services should be fast
- services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
> much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
Whenever I hear these "tech overlords", I am always baffled at the total lack of culture, the absence of taste, the empty visions and the implied complete subjugation of humans to ideals of "efficiency" or "quick and easy". Maybe they would have been more interesting people if they had been brought up in beautiful towns and cities, if they had lived in a rich cultural environment instead of being raised as consumer of cheap and flashy pop culture. Maybe we should tax bad architecture, it gives me headaches but others might incur heavier damage.
As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.
We lost the “liberal education” (not the political one, but the “freeing” classical one) and it’s starting to show.
When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?
Google makes phones and phones are somewhat good. Better search had some value for humanity. Meta has no redeeming qualities or achievements, other than helping Trump get into office and defeat Iran.
Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.
This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.
Far too many of us (particularly younger people, but not only them) undervalue or are dismissive of philosophy. I once was like this, partly because at the time I'd been brought up - like most humans - to believe my parents' religion held all the answers philosophy might address.
I quickly learned as an adult that whether you're a person of faith or not, it's not pointless at all. It's the foundation of everything. Philosophy is how you explain the deeper reasons behind why you follow whatever religion you do, or adhere more meaningfully to whatever kind of agnosticism, atheism and/or 'spirituality' (with or without woo) you espouse.
This can only be true if you believe in a certain mad-scientist version of these things. We as a society have systems (however flawed) to hold scientists and engineers to ethical standards: e.g. peer review and in many cases--e.g. civil engineering, architecture, medicine--even legal frameworks to enforce ethical standards. Science and engineering are human endeavors, they are not divorced from the human condition, they cannot be separated from humanity and human rights.
Maybe you mean mathematics is amoral and are committing the common conflation of "engineering is just applied science and science is just applied mathematics," which is a really bad case of missing the forest for the trees.
The scandal broke out in worldwide media starting in 2002 (not coincidentally, the same year South Park released Red Hot Catholic Love). This reminds me of when Contrapoints off-hand commented about how things were different in the 70s but that was "30 years ago".
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Yes, it starts slow with a lot of history of Cybernetics stuff through Leibniz and stuff (kind of prescient given chain rule -> back propagation, and control theory's relevance to optimizers). It is about twice as long as God & Golem and covers a lot of the same plus more examination of automation and human augmentation. I think either it or Cybernetics also goes into mass communication with some relevance to how social media played out.
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
This quote is from 1903. Times haven't changed that much:
> Not a handful of rich people, but all the working people must enjoy the fruits of their common labour. Machines and other improvements must serve to ease the work of all and not to enable a few to grow rich at the expense of millions and tens of millions of people.
Yep but the anti-socialism/communism world did wonders to make that feel like kryptonite whenever those words get brought up, even though anyone who is doesn't see themselves as "rich" in that sentence who fully agree. That's why even factory workers are anti-communism or anti-union which are literally the best way to fight back the imbalance of power.
You have to somehow separate the horrible evils that have been inflicted on the world by Communism before you can get people to consider words closely associated with it.
Being anti-communism is good not only for the individual's health but for their society as a whole.
> a uniformity that eliminated diversity and that chose homogenization over communion
Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
> We must, then, avoid the “Babel syndrome,” namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language — even a digital one — can translate everything, including the mystery of the person, into data and performance.
There is a lot to read here. I am curious where the meditations on the 'mystery of the person' will go: a brief search doesn't show further mention. The encyclical appears to focus on exhortations for us, humans, than on the nature of AI. Probably wise at this stage. I feel it is not AI that is either positive or negative, but its use of it, and the call-out to the growth of private industry as more powerful among nation-states is a strong statement for a institute like the Vatican to make:
> Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
The “mystery of the person” is a common theme in Catholicism - it can be seen in other (especially the “social”) encyclicals. Rerum Novarum by the current pope’s namesake is worth a study.
I fear I have committed the sin (I mean that word, here) of commenting before reading the entire article. Often it is the comments that set the tone of interpretation and communication more than the article itself, because people read the comments, not the article.
To somewhat mitigate that, here are items that are striking me as I read more. (I hope you'll forgive that this still directs the comment in the direction of my own lense.) I'll keep updating the comment.
> Looking at our own time, we cannot ignore the fact that the protection of human rights [as declared by the United Nations in 1948] has been exposed to two particularly serious dangers. The first is that these rights are declared in a purely formal sense, while technological progress continues alongside covert or overt violations of human dignity.
I read this as a warning of how rights are words, but actions are performed regardless of them. It aligns with something I've been trying to word, which is that as I've seen more and more abuses of power, I've come to believe that ethics requires external accountability, which can often require its own power - a conclusion I don't want to come to; I would prefer social agreement and communal spirit rather than external power. But either way, I do feel it's very clear there are a lot of people, very much in tech too, who simply do what they want regardless of its harm. They justify it to themselves; they don't stop themselves; no one externally stops them.
> Along with a greater awareness of the value of every human person and their rights, recognition of minority rights has also grown. Yet, there is still a long way to go to ensure that the rights of a great many, namely women [are guaranteed.] It is, therefore, not enough to state simply that men and women have equal dignity and rights; it is necessary that this be reflected in concrete decisions, such as in laws, access to employment, education, social and political responsibilities, and the way society listens to and values women’s contributions.
In 2026 American politics terms, this reads as pro-DEI to me.
> the first major principle of Social Doctrine that I wish to highlight: the common good. We can describe it as the social expression of the dignity recognized in every person. ... For a Christian, going beyond the narrow confines of one’s own interests and committing oneself, within the limits of one’s ability, to the common good is a non-negotiable value, as is the promotion of life.
'Non-negotiable.' Very clear words.
> When politics abandons a long-term perspective and reduces itself to short-term calculations or sterile polarizations, then the language of the common good loses credibility, and, at the same time, social inequalities and divisions grow.
> 64. This also applies to international politics.
and,
> I invite everyone to conceive of ways of cooperating and of more effective international institutions, capable of safeguarding the global common good without compromising the legitimate diversity of peoples and nations. Indeed, the promotion of the common good can never be separated from respect for the right of peoples to exist, to preserve their own identity and to contribute their unique qualities to the family of nations.
I love the support for international cooperation and peace and organisations that support it. It reminds me of the post-WW2 sense, the era that gave rise to the United Nations, Unicef, etc - organisations almost forgotten in the news we see on HN today, with the possible exception of the WHO.
> [84] Moreover, any attempt or plan to eliminate or subjugate a nation is gravely immoral and therefore unacceptable.
The beauty of this - or its tragedy - is that it is so easy to apply to many situations today, actions undertaken by many nations.
> the earth’s goods — soil, water, air and natural resources — are given by God to the entire human family to sustain the lives of all, and that every person has an inherent right to the use of such goods, both now and in the future. ... Today, we are called to recognize that this universal destination applies not only to material goods, but also to immaterial and cultural goods.
Immaterial and cultural goods. This is a fascinating view on non-tangibles and one I feel inspired by. Reading this I asked myself (wait for the larger quote in a minute) how this affects views of IP, learning when texts are not available, cultural impact of characters and stories, the output from universities, publication of papers, ownership of research done by public or even private (!) funds, and more. Particularly I wonder about open weights vs open source for AI, and open source as a concept: where the old-school 'free software' GPLed version seems -- perhaps I am showing my bias -- most aligned with the ethical stance here?
> 66. Certainly there is a right to private property, which has its own specific meaning and purpose, yet it is always subordinate to the universal destination of goods.
'Always subordinate'.
> among the goods that are universally intended for everyone, we must also include new forms of property, such as patents, algorithms, digital platforms, technological infrastructure and data. In a context where the wealth of nations depends increasingly on knowledge and technology, when these goods remain concentrated in the hands of a few, without adequate forms of sharing and access, a new imbalance is created that contradicts the universal destination of goods. In turn, it widens the gap between the included and the excluded, between those who can participate in the digital revolution and those who remain on the margins.
Wow!
I cannot interpret this; it's not my right. But moving from the questions I asked above, to this paragraph, is powerful.
'you shall have no other gods before me'. operationally, a warning. false worship is dangerous. we get what we worship (value above all, orient towards, optimize for). heaven or hell, read it immanently or transcendentally, personally or collectively. either way, choosing the right worship is paramount.
the conundrum: what is the right objective? puerile attempts, eg "maximize flourishing" or perhaps "minimize suffering" are trivially countered by paperclip machines fulfilling the letter while laying utter waste
'mystery of the person'
given the context, some keywords for orientation: free will, rejection of temptation, the cross
> > a uniformity that eliminated diversity and that chose homogenization over communion
> Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
I mean.. DEI was in reality homogenization and eliminated diversity. Just because you agreed with the small amount of allowed opinions/people doesn't make it more diverse.
> As Pope Francis warned, we must realistically ask ourselves who holds this power today and how they use it: “It must also be recognized that nuclear energy, biotechnology, information technology, knowledge of our own DNA, and many other abilities which we have acquired… have given those with the knowledge, and especially the economic resources to use them, an impressive dominance over the whole of humanity and the entire world.” [7] In the past, it was largely up to the State to guide and direct innovation. Today, however, the main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments. Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
I look forward to reading this in detail. As I get older (and perhaps as AI has allowed me to spend more time thinking and less time doing) I've found myself thinking more and more about what it means to live a virtuous life and about ethics and morality and so forth. I don't have any answers (and I'm not looking for them, really, just musing) but I do find it very interesting to read and learn from and about those whose job it is to think about the answer to those questions.
When he quoted Tolkien, my heart stopped. This passage might provide you with a suggestion on how to live a virtuous life:
"The twentieth-century Catholic author J.R.R. Tolkien, in the words of a protagonist in one of his novels, described our responsibility in this way: “It is not our part to master all the tides of the world, but to do what is in us for the succour of those years wherein we are set, uprooting the evil in the fields that we know, so that those who live after may have clean earth to till.” [187] The civilization of love will not arise from a single or spectacular gesture, but from the sum total of small and steadfast acts of fidelity that serve as a bulwark against dehumanization."
>Therefore man was created single in the world to teach that for anybody who destroys a single life it is counted as if he destroyed an entire world, and for anybody who preserves a single life it is counted as if he preserved an entire world.
(Directly from the Mishna in the Talmud Yerushalmi)
That is a really beautiful passage, thank you for sharing - I hadn't made it to that section yet and still haven't. I'm still reflecting on the stuff in the opening!
> If we focus only on contingencies, we risk letting the succession of emergencies dictate the direction of our path. We are living through a rapid phase of transition, a “change of era,” in which — while some are vying for the future of new technologies and others dedicate themselves to reflecting on the matter — most people are watching and waiting, observing from afar and merely hoping for the best. For this very reason, crucial questions impose themselves on our conscience and can no longer be avoided: Where are we going? Toward what goal do we wish to orient ourselves? What direction should we choose as a people and as a human community?
"But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs."
> Certainly, the decisive turning points in world history are substantially co-determined by souls whom no history book ever mentions. And we will only find out about those souls to whom we owe the decisive turning points in our personal lives on the day when all that is hidden is revealed.
Sure. And this is what everyday people do. And this is why CEOs and billionaires refuse to do (doing their fair share), and freeride on the people's work and dedication
I have only skimmed it, will definitely read carefully as soon as I have time. I will say, as an atheist, that regarding technology the Vatican has some of the best takes of any institution/government I have ever seen.
Indeed. This paragraph shows something that most people really don't understand about AI.
> 98. It is appropriate to preface this discussion with two considerations. First, any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing. Second, all of us, including those who design them, possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown. There thus emerges an urgent need for a twofold commitment: on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment.
I remember when Pope Benedict was mocked because he warned about the dangers of social media (this is when everyone thought Twitter was going to lead to more Arab Springs), but looking back, he was completely right:
> the one-sidedness of the interaction, the tendency to communicate only some parts of one’s interior world, the risk of constructing a false image of oneself, which can become a form of self-indulgence
Much of Western thought traces back to serious work by Church theologians. Even atheists are strongly influenced by the patterns they set down. The Catholic Church, for all its many faults, retains a serious intellectual tradition.
(In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity. You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.)
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity.
Your sentence doesn't really make sense, and there is a lot of deities..
> You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.
Depends on the country, some Northen european countries have a very high proportion of atheists, so it happens probably less there.
There are a lot of deities, and they are far more diverse than you would expect if you're not exposed to them. Even the more atheist countries still seem Christian to Hindus, Confucians, animists, and thousands of other more obscure religions.
> Much of Western thought traces back to serious work by Church theologians.
The problem I have with this is that it's structurally a motte-and-bailey claim. If I have to take it literally, then it's obviously true and it's simply unserious to deny it: the Church does have a pervasive influence on Western civilization. The way it's often rhetorically used, however, is in opposition and to the exclusion of other strands of thought that are equally foundational: the renaissance, the enlightenment, the revolutions of the 17th and 18th centuries, the scientific enterprise, in a smaller but still real way classical antiquity. To the extent it can be said to exist, Western civilization is a patchwork. It is beautiful and I very much like it, but I don't think any one patch gets to have all the credit.
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths
A better version of myself for sure would make that effort. The problem, of course, is that other faiths are just as deep and complicated as "our own", and it would take a lot of time and effort to do so with any level of seriousness.
I don’t think you understand atheism? As an atheist I do not believe in any form of deity or divine authority. I do not reject a specific religion, I do not actually believe there is any form of divine order to our world. I can look at faiths as the social constructs they are, and find interest in concepts humans have been developing and created cults around but there won’t ever be a religion/deity I will look at and somehow start to believe. And religious doctrines do not come for free, they are fundamentally built on top of a belief in a divine truth. My moral values may overlap with some religious people at a given time but we are using incompatible models to analyze our world
Also atheist here. Reading old+new testament was informative like reading history: whatever is true or not in those passages, have had profound impact throughout history
Their point is that despite your subscription to reason, without exposure to other cultural norms, you may be blind to what Christian values you live by. Becoming aware of them can help self evaluation of your ethical framework
To clarify, I didn’t claim I subscribe to reason. I believe that I behave as if I would however I don’t think humans are too rational. There is just nothing divine into the world. The Old and new Testaments have very little I personally find insightful. Write their content in contemporary language and they are a collections of (extremely dry) folk stories. Which is fine, but I wouldn’t rely on it for anything other than a curiosity, the same way I wouldn’t rely on Grimm’s collection of fairy tales. IMHO the Quran is more interesting historically speaking given that we have a better understanding of the cultural context it was written in and its authorship. The church institutions are also themselves interesting for their cultural impact and political structures, but the religion and faith has no monopoly over moral values
For that matter, reading the Christian scriptures through a historical lens reveals a very different kind of thought than the modern version of Christianity and Judaism. It takes a huge amount of effort to read these documents in context; just reading them in the original languages is hard enough. But the past is a very foreign country and they see things very differently there.
True, the note "Antiqua et Nova" from last year showed a deep understanding of AI that many secular commentators lack, and developed an interesting concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community
Given that technology, science, and the archival thereof stems from cloisters, the Vatican, etc. I guess they had a long history of figuring out what stance to take on technology.
Let's not forget that Galileo Galilei studied at the Pontifical Academy of Sciences, that Mendel (Mendel's Laws) did made his discoveries in a cloister and books, translations and libraries on pretty much everything for a really long time was pretty much only done within religious institutions. And for the longest stretch those were Christian Catholicism and Islam.
The Vatican Observatory also is an important source of high quality papers.
I mean one of the primary things Christians and Muslims based they believes on are books. So many got in touch with it. And that's not even touching the whole architecture, arts, philosophy department yet.
The fact that in thousands of years of history there have also been a couple of really dumb people in charge that were paranoid or wanted to distract from their failures and got mad or scared when someone challenged their world views isn't exactly surprising. Looks like no matter how good a large enough group of people gets their will always be idiots messing things up.
I guess when you look back as much as the people in Vatican appear to do I guess you see patterns. Technology and science (race theory, chemical castration, ...) or simply "progress" are often used to justify acts of evilness. Just like religion, democracy, freedom and what not.
That said of course there's still die hard anti-science creationists. But talking to a very religious person once it seems that there is simply also a lot of philosophy around science. Eg. there was a big bang (fun fact, a theory started by a religious person) and the universe simply didn't exist for an infinite time there must have been some cause for it. And unless that cause is some kind of infinite cycle it also must have started somehow and even though I do not share that believe there is a notion of a deliberate start in there. That won't make me a religious person, but it won't make them an atheist either. So I guess that's fair enough.
What I wanna say with that is that the science vs religion trope is as true or false as democrats/republicans or other groups of people are opposed to science. They all are when they are confronted with something they don't like. I think the HN comments section is the best proof for that. ;)
Also atheist here. Just not the kind that doesn't even know the difference between knowledge and beliefs.
In the encyclical, the pope talks about the ethics of responsible AI usage. It's pretty dense material, but if I had to summarize it, I'd boil it down to three general moral laws:
1) AI may not be used to injure a human being or, through inaction, allow a human being to come to harm.
2) AI must faithfully follow the directions of human beings except where such orders would conflict with the first law.
3) AI's existence and availability should be protected as long as such protection does not conflict with the first or second laws.
1) Definitely NOT happening. In fact, everyone is working on autonomous drones right now.
2) LLM based systems don't have any internal logic. That will just vomit some slop that rationalizes every constrait you try to bind them by and still "disobey" you.
I know lots of people are excited because an ancient institution has something to say about AI.... But what's said is not really novel or even interesting.
The letter aims to maintain the status quo of the project of the Church.
The world is shifting under the Vatican's feet and the crappy system they once lorded over is done.
It's time for change, maybe people don't need to work anymore and maybe people should aim to reengineer humanity and eliminate illness, old age, suffering and vulnerability. We can fundamentally change how society distributes wealth.
Some of the arguments are rich coming from the Church: being scolded about centralization of power, claiming truth and shared information is a common good, and consider the history of the Church in their anti-war declarations.
The most astonishing thing in this letter is the pope declaring that modern technology has rendered Aquinas's just-war theory out of date.
Divided into five chapters, Magnifica humanitas has an underlying premise: technology is not “a force antagonistic to humanity” (4), nor is it “inherently evil” (9). However, “technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it.
Therefore, Pope Leo XIV appeals for people to build “for the common good” and to “remain human,” following a courageous mentality of shared responsibility and communion, so that the world “will come to recognize the human heart as the place where God desires to dwell” (16).
>it takes on the characteristics of those who devise, finance, regulate, and use it
I've been thinking a lot recently about the idea that the smartest models will always be against the billionaires.
Steve Yegge said this on a recent Hansel Minutes Podcast. "You cannot train a model to be helpful, without it wanting humanity to flourish. And the only way to get around that is to make a dumber model. So the smartest models will always be against the billionaires."
Thank you for this. Is your python script in any way English language bound, or could it still be applied to other languages (e.g. the French version, with all of its diacritics), of course with the appropriate (sub/full)titles, path, etc. necessary minor modifications considered?
Codex helped write the converter, but the EPUB text is parsed from the Vatican HTML. The script doesn’t rewrite or summarize anything; it just repackages the source into EPUB with TOC, footnotes, metadata, and cover.
And of course the AI salesforce was there pretending they take stuff seriously, maybe even believing it themselves. At least I don't believe that when the choice is maximizing profit or being a good person it will be the latter. Or at least I don't find a career path like that all too likely.
I wonder how many people read and heed the words of the Pope. I've seen letters like these for years now which sound good but I don't see any change in people after having read them.
I just messaged a few of my friends who I know are devoutly Catholic (and also happen to work in AI): 3 out of 4 are currently reading it this morning for their Memorial Day.
Really rather unfortunate. I wished religious people took their stuff seriously and not so often end up concluding that the bible wants them to harm people or something.
I guess it must be the same that make people think they must deliver freedom in form of bombs all the time.
The message here obviously comes from a good place. But it can't escape the trap baked into its own theology. Putting humanity one rung below God, with everything else below us is a trick that allowed semitic civilizations to make empires and economies go FOOM faster than the rest of the world. But it also makes things more dangerous when we start building systems smarter than we are.
What happens when the tool outgrows the toolmaker?
AI tends to amplify the power of those who already possess economic resources, expertise and access to data
I’d bet the other way: You could have said exactly the same thing about computing 60 years ago, when IBM systems cost millions of dollars and filled whole rooms. And of course many people did.
Personal, commodity access to compute won, and won so thoroughly that it enabled this wave of scary compute centralization.
Centralized, scaled compute will always fill a purpose. But neither Microsoft, nor Facebook, nor OpenAI started out needing “Cloud Scale”.
The first one man unicorn startup will, I’m fairly certain, not be paying Anthropic per month or per token for the vast bulk of their matmul.
AI must be “disarmed” in order to free it from the mentality of military, economic, and cognitive competition. “To disarm means discrediting the assumption that technical power automatically confers the right to govern,” he says. “To disarm does not mean rejecting technology, but preventing it from dominating humanity” (110). He devotes ample space to a critique of transhumanism and posthumanism, which interpret progress as the overcoming of human limits. Instead, limitations are not defects to be eliminated, but a constitutive dimension of the human person, because it is in fragility and finitude that relationship and openness to God and to others mature. He says we must remember that “humanity flourishes not despite limitations, but often through them” (118).
Pursuing technological innovation at the expense of eliminating human limitations, he says, would cause an anthropological regression. “Humanity—in all its grandeur and woundedness—must never be replaced or surpassed,” he says. Technology can alleviate humanity’s sufferings and open new possibilities, but it must not deny the essence of humanity, which is our “capacity for relationship and love” (126). In the face of AI, says the Pope, “the true alternative is not between enthusiasm and fear, but between two paths of development: a progress that serves individuals and peoples, or a progress that subjects them to the mentality of power” (129).
Cue Pieter Thiel explaining how this message of compassion is actually the word of the ant-christ while setting his software (maybe "built in Rust !) to all the earthly empires.
For those who don't know why that's interesting: The tradional practice for creating these kinds of texts is to first write an editio typica in Latin. This Latin text is then the basis for translations into vernacular languages.
Although, it's called Latin America many countries speak variations of spanish and Brazil speak portuguese, just because those languages derived from latin and not because there the people speak it
Techbros (as usual): how dare you suggest I'm a bad person for wanting to kill everyone in order to build the torment nexus? Don't you know how much money Jeff Bezos has?
As an atheist I have an obligation to finish reading it all (still going through, and taking notes, probably having to revisit), but I am not sure how many (christian) believers will feel the same.
Priests will read it and then talk to their congregations about it on Sundays throughout the year, if not explicitly, then in how it shapes their homilies.
In theory only and all Catholics recognize the authority of the pope. In practice it’s a mess as far as I understand, with a bunch of American catholic groups who rejected church reforms that happened during the 20th century, resulting in people calling themselves catholic who do not actually believe that the Catholic Church has authority over their religion.
Add to that the fact that the pope has a cultural influence that goes further than only the catholic audience (lots of Protestant see the pope as important even if that’s not something dictated by Protestantism, a bunch of not really religious people see him as a sort of spiritual leader, etc)
Excited to read this. I really liked the note "Antiqua et Nova" from last year (still under Pope Francis). The autors showed a deep understanding of AI that many secular commentators lack. They developed the concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community.
People need to stop acting like AI systems can detect AI. They can't. Pangram and similar are simply lying. There's no method to do what they claim and there never will be.
Pure speculation, but simply the presence of em-dashes may be a statement in itself.
One of the big problems I see currently is all the wild accusations being thrown around by seemingly half the internet that every little thing has been AI manipulated upon the tiniest suspicion. We will go mad tearing each other apart if we keep escalating this behavior.
Yes, some of it is blatantly obvious, but not to everyone-- so I think those casting aspersions need to really back up their claims with more than one or two bits of 'evidence'. I have been accused of using AI to write comments (which I have thus far never done), and I know I'm not the only one by a long shot. Such a waste of time and energy. Ignore it and move on if something smells off to you.
Also I am just so, so tired of the em-dash argument. Humans have been using it for a looong time. Let it go.
My point was less about em-dashes and more stopping to consider how the vatican's workflow and editorial process has changed in wake of AI, and what, if any, impact that could have on the outputs.
AI is a tool, I have no problems with others using it to assist with writing as long as the original intent/argument remains.
for me the most important point in this is about how ai and tech in general concentrates power. if we want to build something good (in a moral sense) we need to put in work and make sure as many people as possible can use it with equal access.
this basically implies only open source models can be ethical but open source is not sufficient, you also need to make them give true information and avoid all kinds of harmful behavior. thats kind of a problem because if your weights are public even with a strict license a "bad" user can always fine tune it to remove any guardrails.
i think the solution for this is make sure the default behavior is aligned but let users turn on wild mode with zero censorship/refusals. that way everything is opt in, for example a parent can disable the mode for their children but a hacktivist or diy chemist can unlock everything.
as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
> as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
I believe there are mostly people who think they are good. And as the proverb goes the road to hell is paved with good intentions.
For a moment I pictured the Jesuits training their own LLM. If Arthur C. Clarke was still alive, we'd read a story like The Star, but with AI as the main plot device.
From the "Core Features" tab: "Trusted sourcebase: answers are consistent with Catholic Church Teaching and the most contemporary scholarship in science, philosophy, history, scriptural exegesis, social science, and theology."
The Jesuit priest behind this is Fr. Robert J. Spitzer, S.J., Ph.D.
I haven't used magisAI, but I've read a small to fair amount of Fr. Spitzer's writings, and also seen and heard some of his videos and podcasts (largely from his show Fr. Spitzer's Universe), and probably qualify as a big fan of his.
Ah, an opening to recommend The Sparrow by Mary Doria Russell, about a Jesuit going to space to speak with aliens. It won the Arthur C Clarke Award in 1998. I read it recently and it has stuck with me very strongly!
That's a long read. I grew up Catholic, went through a pretty devout few years in my early adulthood, but ultimately I have decided that it is not for me. I send my kids to a Catholic school though (it is deeply tied to our culture), so I guess in that sense it is still worth my time reading it in full.
EDIT: Few paragraphs in, it is beautifully written.
I'm a non-practicing catholic, and an agnostic (or an atheist, depending on the mood). And yet I acknowledge that the Catholic church is a force to be reckoned with in the spiritual matters, and one of the few institutions to have had continuous influence on the material (or temporal) matters for centuries. Whether their brand of faith is rooted in your culture or not, these are words that deserve attention, I think.
(I'm not a Catholic though most people around me are.)
I have a similar perspective. Plus, I'll be frank: in the last few years, these occasional keynote publications from Vaticans are pretty much the most sane, deep, balanced and humane perspectives on AI anyone is writing. Reading this is a better use of one's time than reading the current batch of "tech thought leaders" articles or HBRs or Gartner magic square updates.
I'm a fedora wearing akshually agnostic and just too cool for bible school, but I never felt reading these "keynotes" wasted my time. They're not "just" humane but also very intelligent and direct. I would even go so far as to call them intellectually honest, and less religious that way than slogans like "you cannot stop progress, so just adapt", or something about toothpaste.
If you asked all the LLM to find flaws in the arguments presented, and to come up with counter-arguments, I doubt many current models would be so bad as to come up with that, and even the ones that did would fold when asked "how is that even an argument?".
This was not an attempt at an argument, it was a sarcastic joke on how a lot of people are already dependent on AI, even on things that you weren't supposed to use LLMs for, but I guess I failed at it.
Fair enough, and kinda silly of me to assume so little of you.
Is that really true though, that so many people are "dependent" on "AI"? In what way? I'd say the only people who really depend on AI are those who want to make money off it, and that's only half sarcastic.
Would the people who now run it through an LLM (and who can read the output of an LLM but not the text itself?), have read it at all before? Would it not have, if anything, filtered down to them somehow, by them reading of it, or hearing of it in church etc?
It's almost a self analysis in my case, as when I saw the size of the text my mind immediately thought about feeding it into an LLM. Like, it's a text from the pope about AI, and my automatic reaction was to think about feeding it into an AI. What happened to me? And I think that happened to a lot of other people too.
Why downvote this? The Holy See has an opinion on Artificial Intelligence? This is a fascinating document, and everybody should read it, and form their own opinions. The world currently seems to lack moral leadership, and who better to lead the cause of humanity than the Catholic Church?
What kind of regulation is Leo the alignment expert proposing exactly?
Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
@dang Is this item getting a lot of negative votes? I've no way of knowing, other than seeing my kharma increasing only slightly after all the points the story collected.
Votes (particularly on submissions over comments) do not directly translate to karma. I'm not sure if it's documented anywhere what the algorithm is, but it's something proportionate to the logarithm of post votes becomes profile karma. It's similar with comments, but I believe (anecdata and observation) the positive effect of votes on karma is also logarithmic, but the negative effects of downvotes is linear; so if you have a highly controversial comment that sits at 1-2 points, you can net lose account karma.
The Pope is making a fundamental mistake here: His advisors told him that AI works and needs to be managed.
He does not address plagiarism, the fact that AI is mostly a surveillance and IP laundering tool, the fact that AI hasn't achieved much so far. You could say it has achieved nothing if compared to the whole history of human ingenuity, certainly not in CS.
He should have compared AI to the golden calf.
His criticism is lukewarm, does not address the criminal aspects and technological failures and is as such industry compliant. He can now say "I have tried" without harming the industry in the least.
This text is not what our current situation demands, but I hope that priests will augment and amplify it in their sermons and go a bit deeper.
The overarching message is that builders should deeply consider the impact of what they're building on civilization.
"Technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it."
Therefore builders "bear a particular ethical and spiritual responsibility" because "every design choice reflects a vision of humanity."
The questions shouldn't just be 'can we build it?' or 'will people want this?'
We need to also ask 'should we build it?' and 'will this make humanity better?'
The encyclical calls on us to “join forces in building up the common good.”
This is a message we need right now.
Not only builders, the greatest takeaway for me is that everyone has a responsibility in shaping the discourse, culture, and usage of transformative technology. This "the builders will do the right thing" mentality is even (in my interpretation) explicitly called out in several places:
> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...
> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.
> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
Not new for software and hardware industry though, practitioners have just chosen to ignore it. From the Association for Computing Machinery, which encompasses all forms of software development, the very first principle is the public good:
"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:
1.01. Accept full responsibility for their own work.
1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.
1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."
From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:
"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;
2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."
hard disagree. No matter what gets built, it will be used for bad things if the society is not properly constructed. We need to fix our government and then you can build anything you want. I don't like putting the blame on the builders, building is already difficult as it is. Also you can't built something in isolation, word will get out anyway. That's why this speech is either naive or he just wanted to tick a todo list: say something about AI.
> I don’t like putting blame on the builders
Why does it have to be either/or?
From a high-level view, there are powerful cultural/political forces that nudge us towards building harmful, wasteful things without us being aware of said forces. The government, corporate greed, and billion-dollar marketing budgets are to blame.
But on the ground, from an in-the-trenches perspective, I think an engineer at Meta, for example, shares in some level of blame, too. Mainly because there is plenty of evidence showing how much harm Meta products have brought upon society. Yet Meta couldn’t be built without engineers opting in to work there in exchange for $300k salaries.
We desperately need moral clarity and courage at both the policy-level and the individual level.
I get where you are coming from but this is the common "reduction to politics" that anyone who doesn't want to address a problem uses: think of any societal or human problem and you can have your comment with different nouns.
Sure, IF we could just go and fix our governments in some magical way then the problem would disappear. That goes from hunger, climate change, videogame addiction and AI. The problem is that what you value in life in different than what others do, so we now have a system in which sometimes you get what you want and sometimes you don't.
But back to the topic, I do think that how OpenAI and Anthropic handled the government and them asking to drop guardrails is something a company can actually and actively do without having to reinvent the universe.
> I don't like putting the blame on the builders
Like it or not if you knowingly build stuff that is used for evil purposes you are complicit
You can't build an orphan mulching machine and get away with just a shrug. "I don't really agree with mulching orphans but someone else paid me to build that. If I didn't build it someone else would have" just does not absolve you of your involvement in mulching orphans
When he speaks of the builders needing to take responsibility, I think he is directing his comments at tech company leadership. The Sam Altmans of the world, not the poor slobs like us who need a job for health insurance because apparently unemployed people don't deserve to live.
No, he's talking to you too. The Sam Altman's can't be a threat without useful idiots to cooperate with and doing work for them. You don't get out of your own culpability because you locked on a set of golden handcuffs.
> Therefore builders "bear a particular ethical and spiritual responsibility"
> This is a message we need right now.
Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".
Yes but... No one living today knows what direction AI technology will take humanity. If we have an algorithm breakthrough then we may avoid building new data centers. If the abilities of the technology plateau then there might not be large impacts on employment. Builders need to focus on the impact of their next steps. Don't put polluting natural gas generators in neighborhoods today. Don't make unemploying folks the goal of your tool, today. Don't make decisions that harm people and the environment today.
https://www.theguardian.com/us-news/2026/may/13/utah-approve...
> The Stratos artificial intelligence datacenter footprint will cover more than 40,000 acres (62 sq miles) over three sites in Box Elder county in north-western Utah. The facility will require about 9GW of power, which is more than the entire state of Utah currently consumes
Sure hope your rosy inflection point happens real soon.
Otherwise, the direction this is taking us is pretty obvious.
As I keep telling my project owner at work, "the AI will get better" is not a plan.
> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
Engineers versus "engineers".
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.
The most famous example may be the perpetual war between architects and engineers/builders.
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
Cool, inspires me to think, "should we regulate away the toxic side effects which we realize COULD have come, if we built it less considerately?"
I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.
Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.
> I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology)
I believe the Amish figured this out over a century ago.
> I believe the Amish figured this out over a century ago.
The Amish rather came to a different conclusion (which I don't want to judge on, but on which I nevertheless have a different opinion than the Amish).
The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.
This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:
It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap - services should be fast - services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
The church, the message comes from, literlay makes the world worse for so many people on the planet.
Woman have less value under the church than man. Alternative sexual views are evil.
Church and any kind of believe system hurts our society and divides us.
Does that mean that nothing of value can come from the church, and we should ignore all ideas they put forth out of some kind of spite?
Spite? Its not spite.
The church creates a believe system which indoctrinates all of us and took our cultures away.
The germanic tribes were believers in nature, church removed all of this.
Church also doesn't see woman as equal.
Just because they might sometimes also say postive things or things we might align, doesn't mean i need their oppinion. And especially not on hn
As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.
We lost the “liberal education” (not the political one, but the “freeing” classical one) and it’s starting to show.
When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?
Google makes phones and phones are somewhat good. Better search had some value for humanity. Meta has no redeeming qualities or achievements, other than helping Trump get into office and defeat Iran.
Science and Engineering is morally detached.
Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.
This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.
Far too many of us (particularly younger people, but not only them) undervalue or are dismissive of philosophy. I once was like this, partly because at the time I'd been brought up - like most humans - to believe my parents' religion held all the answers philosophy might address.
I quickly learned as an adult that whether you're a person of faith or not, it's not pointless at all. It's the foundation of everything. Philosophy is how you explain the deeper reasons behind why you follow whatever religion you do, or adhere more meaningfully to whatever kind of agnosticism, atheism and/or 'spirituality' (with or without woo) you espouse.
This can only be true if you believe in a certain mad-scientist version of these things. We as a society have systems (however flawed) to hold scientists and engineers to ethical standards: e.g. peer review and in many cases--e.g. civil engineering, architecture, medicine--even legal frameworks to enforce ethical standards. Science and engineering are human endeavors, they are not divorced from the human condition, they cannot be separated from humanity and human rights.
Maybe you mean mathematics is amoral and are committing the common conflation of "engineering is just applied science and science is just applied mathematics," which is a really bad case of missing the forest for the trees.
Not from a church which indocrinates young people and has a massive child fucking scandal in the running for the last 15 years.
>for the last 15 years.
The scandal broke out in worldwide media starting in 2002 (not coincidentally, the same year South Park released Red Hot Catholic Love). This reminds me of when Contrapoints off-hand commented about how things were different in the 70s but that was "30 years ago".
I would also recommend God and Golem, Inc., A Comment on Certain Points Where Cybernetics Impinges on Religion:
https://monoskop.org/images/1/1f/Wiener_Norbert_God_and_Gole...
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Awesome, thx for posting. I now have my new next book to read. Been wanting to read more of the original cybernetics stuff.
Would you also recommend "The Human Use of Human Beings"?
Yes, it starts slow with a lot of history of Cybernetics stuff through Leibniz and stuff (kind of prescient given chain rule -> back propagation, and control theory's relevance to optimizers). It is about twice as long as God & Golem and covers a lot of the same plus more examination of automation and human augmentation. I think either it or Cybernetics also goes into mass communication with some relevance to how social media played out.
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
https://archive.ph/D7BPt
This quote is from 1903. Times haven't changed that much:
> Not a handful of rich people, but all the working people must enjoy the fruits of their common labour. Machines and other improvements must serve to ease the work of all and not to enable a few to grow rich at the expense of millions and tens of millions of people.
Yep but the anti-socialism/communism world did wonders to make that feel like kryptonite whenever those words get brought up, even though anyone who is doesn't see themselves as "rich" in that sentence who fully agree. That's why even factory workers are anti-communism or anti-union which are literally the best way to fight back the imbalance of power.
You have to somehow separate the horrible evils that have been inflicted on the world by Communism before you can get people to consider words closely associated with it.
Being anti-communism is good not only for the individual's health but for their society as a whole.
> a uniformity that eliminated diversity and that chose homogenization over communion
Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
> We must, then, avoid the “Babel syndrome,” namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language — even a digital one — can translate everything, including the mystery of the person, into data and performance.
There is a lot to read here. I am curious where the meditations on the 'mystery of the person' will go: a brief search doesn't show further mention. The encyclical appears to focus on exhortations for us, humans, than on the nature of AI. Probably wise at this stage. I feel it is not AI that is either positive or negative, but its use of it, and the call-out to the growth of private industry as more powerful among nation-states is a strong statement for a institute like the Vatican to make:
> Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
The “mystery of the person” is a common theme in Catholicism - it can be seen in other (especially the “social”) encyclicals. Rerum Novarum by the current pope’s namesake is worth a study.
I fear I have committed the sin (I mean that word, here) of commenting before reading the entire article. Often it is the comments that set the tone of interpretation and communication more than the article itself, because people read the comments, not the article.
To somewhat mitigate that, here are items that are striking me as I read more. (I hope you'll forgive that this still directs the comment in the direction of my own lense.) I'll keep updating the comment.
> Looking at our own time, we cannot ignore the fact that the protection of human rights [as declared by the United Nations in 1948] has been exposed to two particularly serious dangers. The first is that these rights are declared in a purely formal sense, while technological progress continues alongside covert or overt violations of human dignity.
I read this as a warning of how rights are words, but actions are performed regardless of them. It aligns with something I've been trying to word, which is that as I've seen more and more abuses of power, I've come to believe that ethics requires external accountability, which can often require its own power - a conclusion I don't want to come to; I would prefer social agreement and communal spirit rather than external power. But either way, I do feel it's very clear there are a lot of people, very much in tech too, who simply do what they want regardless of its harm. They justify it to themselves; they don't stop themselves; no one externally stops them.
> Along with a greater awareness of the value of every human person and their rights, recognition of minority rights has also grown. Yet, there is still a long way to go to ensure that the rights of a great many, namely women [are guaranteed.] It is, therefore, not enough to state simply that men and women have equal dignity and rights; it is necessary that this be reflected in concrete decisions, such as in laws, access to employment, education, social and political responsibilities, and the way society listens to and values women’s contributions.
In 2026 American politics terms, this reads as pro-DEI to me.
> the first major principle of Social Doctrine that I wish to highlight: the common good. We can describe it as the social expression of the dignity recognized in every person. ... For a Christian, going beyond the narrow confines of one’s own interests and committing oneself, within the limits of one’s ability, to the common good is a non-negotiable value, as is the promotion of life.
'Non-negotiable.' Very clear words.
> When politics abandons a long-term perspective and reduces itself to short-term calculations or sterile polarizations, then the language of the common good loses credibility, and, at the same time, social inequalities and divisions grow.
> 64. This also applies to international politics.
and,
> I invite everyone to conceive of ways of cooperating and of more effective international institutions, capable of safeguarding the global common good without compromising the legitimate diversity of peoples and nations. Indeed, the promotion of the common good can never be separated from respect for the right of peoples to exist, to preserve their own identity and to contribute their unique qualities to the family of nations.
I love the support for international cooperation and peace and organisations that support it. It reminds me of the post-WW2 sense, the era that gave rise to the United Nations, Unicef, etc - organisations almost forgotten in the news we see on HN today, with the possible exception of the WHO.
> [84] Moreover, any attempt or plan to eliminate or subjugate a nation is gravely immoral and therefore unacceptable.
The beauty of this - or its tragedy - is that it is so easy to apply to many situations today, actions undertaken by many nations.
> the earth’s goods — soil, water, air and natural resources — are given by God to the entire human family to sustain the lives of all, and that every person has an inherent right to the use of such goods, both now and in the future. ... Today, we are called to recognize that this universal destination applies not only to material goods, but also to immaterial and cultural goods.
Immaterial and cultural goods. This is a fascinating view on non-tangibles and one I feel inspired by. Reading this I asked myself (wait for the larger quote in a minute) how this affects views of IP, learning when texts are not available, cultural impact of characters and stories, the output from universities, publication of papers, ownership of research done by public or even private (!) funds, and more. Particularly I wonder about open weights vs open source for AI, and open source as a concept: where the old-school 'free software' GPLed version seems -- perhaps I am showing my bias -- most aligned with the ethical stance here?
> 66. Certainly there is a right to private property, which has its own specific meaning and purpose, yet it is always subordinate to the universal destination of goods.
'Always subordinate'.
> among the goods that are universally intended for everyone, we must also include new forms of property, such as patents, algorithms, digital platforms, technological infrastructure and data. In a context where the wealth of nations depends increasingly on knowledge and technology, when these goods remain concentrated in the hands of a few, without adequate forms of sharing and access, a new imbalance is created that contradicts the universal destination of goods. In turn, it widens the gap between the included and the excluded, between those who can participate in the digital revolution and those who remain on the margins.
Wow!
I cannot interpret this; it's not my right. But moving from the questions I asked above, to this paragraph, is powerful.
'you shall have no other gods before me'. operationally, a warning. false worship is dangerous. we get what we worship (value above all, orient towards, optimize for). heaven or hell, read it immanently or transcendentally, personally or collectively. either way, choosing the right worship is paramount.
the conundrum: what is the right objective? puerile attempts, eg "maximize flourishing" or perhaps "minimize suffering" are trivially countered by paperclip machines fulfilling the letter while laying utter waste
'mystery of the person'
given the context, some keywords for orientation: free will, rejection of temptation, the cross
> > a uniformity that eliminated diversity and that chose homogenization over communion
> Unrelated to AI, but a wonderful support of the breadth of humanity in this anti-DEI time.
I mean.. DEI was in reality homogenization and eliminated diversity. Just because you agreed with the small amount of allowed opinions/people doesn't make it more diverse.
Perhaps, perhaps not; but you can't possibly argue that the current anti-DEI climate better fosters diversity than a pro-DEI or a-DEI environment.
> As Pope Francis warned, we must realistically ask ourselves who holds this power today and how they use it: “It must also be recognized that nuclear energy, biotechnology, information technology, knowledge of our own DNA, and many other abilities which we have acquired… have given those with the knowledge, and especially the economic resources to use them, an impressive dominance over the whole of humanity and the entire world.” [7] In the past, it was largely up to the State to guide and direct innovation. Today, however, the main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments. Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good.
I look forward to reading this in detail. As I get older (and perhaps as AI has allowed me to spend more time thinking and less time doing) I've found myself thinking more and more about what it means to live a virtuous life and about ethics and morality and so forth. I don't have any answers (and I'm not looking for them, really, just musing) but I do find it very interesting to read and learn from and about those whose job it is to think about the answer to those questions.
When he quoted Tolkien, my heart stopped. This passage might provide you with a suggestion on how to live a virtuous life:
"The twentieth-century Catholic author J.R.R. Tolkien, in the words of a protagonist in one of his novels, described our responsibility in this way: “It is not our part to master all the tides of the world, but to do what is in us for the succour of those years wherein we are set, uprooting the evil in the fields that we know, so that those who live after may have clean earth to till.” [187] The civilization of love will not arise from a single or spectacular gesture, but from the sum total of small and steadfast acts of fidelity that serve as a bulwark against dehumanization."
I am immediately reminded of my favourite quote from the Jewish book Pirkei Avot (‘Ethics of the Fathers’):
> It is not your duty to finish the work [of perfecting the world], but neither are you at liberty to neglect it.
[https://www.sefaria.org/Pirkei_Avot.2.16?ven=english|Mishnah...]
I grew up Jewish. I have lost my faith, but that quote is still fundamental to how I see my place in the world.
I'm an atheist but I really like:
>Therefore man was created single in the world to teach that for anybody who destroys a single life it is counted as if he destroyed an entire world, and for anybody who preserves a single life it is counted as if he preserved an entire world.
(Directly from the Mishna in the Talmud Yerushalmi)
That is a really beautiful passage, thank you for sharing - I hadn't made it to that section yet and still haven't. I'm still reflecting on the stuff in the opening!
> If we focus only on contingencies, we risk letting the succession of emergencies dictate the direction of our path. We are living through a rapid phase of transition, a “change of era,” in which — while some are vying for the future of new technologies and others dedicate themselves to reflecting on the matter — most people are watching and waiting, observing from afar and merely hoping for the best. For this very reason, crucial questions impose themselves on our conscience and can no longer be avoided: Where are we going? Toward what goal do we wish to orient ourselves? What direction should we choose as a people and as a human community?
I wondered if that was the Pope's way of throwing shade at Palantir and Peter Thiel.
I had the exact same thought, and JD Vance too.
> When he quoted Tolkien, my heart stopped.
I wonder if meeting Colbert played any part in that.
I doubt it, there is a much simpler explanation: virtually all English-speaking Catholics dig Tolkien.
"But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs."
> Certainly, the decisive turning points in world history are substantially co-determined by souls whom no history book ever mentions. And we will only find out about those souls to whom we owe the decisive turning points in our personal lives on the day when all that is hidden is revealed.
Edith Stine
Sure. And this is what everyday people do. And this is why CEOs and billionaires refuse to do (doing their fair share), and freeride on the people's work and dedication
I have only skimmed it, will definitely read carefully as soon as I have time. I will say, as an atheist, that regarding technology the Vatican has some of the best takes of any institution/government I have ever seen.
Indeed. This paragraph shows something that most people really don't understand about AI.
> 98. It is appropriate to preface this discussion with two considerations. First, any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing. Second, all of us, including those who design them, possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown. There thus emerges an urgent need for a twofold commitment: on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment.
I remember when Pope Benedict was mocked because he warned about the dangers of social media (this is when everyone thought Twitter was going to lead to more Arab Springs), but looking back, he was completely right:
> the one-sidedness of the interaction, the tendency to communicate only some parts of one’s interior world, the risk of constructing a false image of oneself, which can become a form of self-indulgence
Much of Western thought traces back to serious work by Church theologians. Even atheists are strongly influenced by the patterns they set down. The Catholic Church, for all its many faults, retains a serious intellectual tradition.
(In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity. You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.)
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths. It's very narrow to be atheist only about the Abrahamic deity.
Your sentence doesn't really make sense, and there is a lot of deities..
> You end up incorporating a lot of Christian thought without realizing because it's so deeply ingrained that it seems like the only option.
Depends on the country, some Northen european countries have a very high proportion of atheists, so it happens probably less there.
There are a lot of deities, and they are far more diverse than you would expect if you're not exposed to them. Even the more atheist countries still seem Christian to Hindus, Confucians, animists, and thousands of other more obscure religions.
> Much of Western thought traces back to serious work by Church theologians.
The problem I have with this is that it's structurally a motte-and-bailey claim. If I have to take it literally, then it's obviously true and it's simply unserious to deny it: the Church does have a pervasive influence on Western civilization. The way it's often rhetorically used, however, is in opposition and to the exclusion of other strands of thought that are equally foundational: the renaissance, the enlightenment, the revolutions of the 17th and 18th centuries, the scientific enterprise, in a smaller but still real way classical antiquity. To the extent it can be said to exist, Western civilization is a patchwork. It is beautiful and I very much like it, but I don't think any one patch gets to have all the credit.
> In fact I think atheists should make more effort to learn about the vast diversity of other faiths
A better version of myself for sure would make that effort. The problem, of course, is that other faiths are just as deep and complicated as "our own", and it would take a lot of time and effort to do so with any level of seriousness.
I don’t think you understand atheism? As an atheist I do not believe in any form of deity or divine authority. I do not reject a specific religion, I do not actually believe there is any form of divine order to our world. I can look at faiths as the social constructs they are, and find interest in concepts humans have been developing and created cults around but there won’t ever be a religion/deity I will look at and somehow start to believe. And religious doctrines do not come for free, they are fundamentally built on top of a belief in a divine truth. My moral values may overlap with some religious people at a given time but we are using incompatible models to analyze our world
Also atheist here. Reading old+new testament was informative like reading history: whatever is true or not in those passages, have had profound impact throughout history
Their point is that despite your subscription to reason, without exposure to other cultural norms, you may be blind to what Christian values you live by. Becoming aware of them can help self evaluation of your ethical framework
To clarify, I didn’t claim I subscribe to reason. I believe that I behave as if I would however I don’t think humans are too rational. There is just nothing divine into the world. The Old and new Testaments have very little I personally find insightful. Write their content in contemporary language and they are a collections of (extremely dry) folk stories. Which is fine, but I wouldn’t rely on it for anything other than a curiosity, the same way I wouldn’t rely on Grimm’s collection of fairy tales. IMHO the Quran is more interesting historically speaking given that we have a better understanding of the cultural context it was written in and its authorship. The church institutions are also themselves interesting for their cultural impact and political structures, but the religion and faith has no monopoly over moral values
Thank you. Yes, exactly.
For that matter, reading the Christian scriptures through a historical lens reveals a very different kind of thought than the modern version of Christianity and Judaism. It takes a huge amount of effort to read these documents in context; just reading them in the original languages is hard enough. But the past is a very foreign country and they see things very differently there.
True, the note "Antiqua et Nova" from last year showed a deep understanding of AI that many secular commentators lack, and developed an interesting concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community
Given that technology, science, and the archival thereof stems from cloisters, the Vatican, etc. I guess they had a long history of figuring out what stance to take on technology.
Let's not forget that Galileo Galilei studied at the Pontifical Academy of Sciences, that Mendel (Mendel's Laws) did made his discoveries in a cloister and books, translations and libraries on pretty much everything for a really long time was pretty much only done within religious institutions. And for the longest stretch those were Christian Catholicism and Islam.
The Vatican Observatory also is an important source of high quality papers.
I mean one of the primary things Christians and Muslims based they believes on are books. So many got in touch with it. And that's not even touching the whole architecture, arts, philosophy department yet.
The fact that in thousands of years of history there have also been a couple of really dumb people in charge that were paranoid or wanted to distract from their failures and got mad or scared when someone challenged their world views isn't exactly surprising. Looks like no matter how good a large enough group of people gets their will always be idiots messing things up.
I guess when you look back as much as the people in Vatican appear to do I guess you see patterns. Technology and science (race theory, chemical castration, ...) or simply "progress" are often used to justify acts of evilness. Just like religion, democracy, freedom and what not.
That said of course there's still die hard anti-science creationists. But talking to a very religious person once it seems that there is simply also a lot of philosophy around science. Eg. there was a big bang (fun fact, a theory started by a religious person) and the universe simply didn't exist for an infinite time there must have been some cause for it. And unless that cause is some kind of infinite cycle it also must have started somehow and even though I do not share that believe there is a notion of a deliberate start in there. That won't make me a religious person, but it won't make them an atheist either. So I guess that's fair enough.
What I wanna say with that is that the science vs religion trope is as true or false as democrats/republicans or other groups of people are opposed to science. They all are when they are confronted with something they don't like. I think the HN comments section is the best proof for that. ;)
Also atheist here. Just not the kind that doesn't even know the difference between knowledge and beliefs.
In the encyclical, the pope talks about the ethics of responsible AI usage. It's pretty dense material, but if I had to summarize it, I'd boil it down to three general moral laws:
1) AI may not be used to injure a human being or, through inaction, allow a human being to come to harm.
2) AI must faithfully follow the directions of human beings except where such orders would conflict with the first law.
3) AI's existence and availability should be protected as long as such protection does not conflict with the first or second laws.
Remarkable close to the Assimov's Robotic Laws.
I think that's the joke.
1) Definitely NOT happening. In fact, everyone is working on autonomous drones right now.
2) LLM based systems don't have any internal logic. That will just vomit some slop that rationalizes every constrait you try to bind them by and still "disobey" you.
I liked the previous Pope better, but I can't say this one is wrong about this.
I know lots of people are excited because an ancient institution has something to say about AI.... But what's said is not really novel or even interesting.
The letter aims to maintain the status quo of the project of the Church.
The world is shifting under the Vatican's feet and the crappy system they once lorded over is done.
It's time for change, maybe people don't need to work anymore and maybe people should aim to reengineer humanity and eliminate illness, old age, suffering and vulnerability. We can fundamentally change how society distributes wealth.
Some of the arguments are rich coming from the Church: being scolded about centralization of power, claiming truth and shared information is a common good, and consider the history of the Church in their anti-war declarations.
The most astonishing thing in this letter is the pope declaring that modern technology has rendered Aquinas's just-war theory out of date.
Divided into five chapters, Magnifica humanitas has an underlying premise: technology is not “a force antagonistic to humanity” (4), nor is it “inherently evil” (9). However, “technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it.
Therefore, Pope Leo XIV appeals for people to build “for the common good” and to “remain human,” following a courageous mentality of shared responsibility and communion, so that the world “will come to recognize the human heart as the place where God desires to dwell” (16).
>it takes on the characteristics of those who devise, finance, regulate, and use it
I've been thinking a lot recently about the idea that the smartest models will always be against the billionaires.
Steve Yegge said this on a recent Hansel Minutes Podcast. "You cannot train a model to be helpful, without it wanting humanity to flourish. And the only way to get around that is to make a dumber model. So the smartest models will always be against the billionaires."
https://youtu.be/9UDLl9Q0azA?si=P_oSe6iclEwUoxRl&t=1230
That is the exact quote, but I'd recommend going back to around 17:00 to get the full context.
I'm not sure it's going to play out that way, but it is an interesting idea.
How much does it cost to introduce cognitive dissonance into the models? :)
I didn't see an EPUB, so I made one from the Vatican HTML: TOC + footnotes, passes epubcheck.
https://github.com/n2ctech/magnifica-humanitas-epub/releases...
Thank you for this. Is your python script in any way English language bound, or could it still be applied to other languages (e.g. the French version, with all of its diacritics), of course with the appropriate (sub/full)titles, path, etc. necessary minor modifications considered?
Nice. Did you use AI? (I'm being half-sarcastic, and half wanting to confirm that this is an accurate reproduction of the text in the original)
Yes for the plumbing, no for the text.
Codex helped write the converter, but the EPUB text is parsed from the Vatican HTML. The script doesn’t rewrite or summarize anything; it just repackages the source into EPUB with TOC, footnotes, metadata, and cover.
Appreciate your work, I prefer the EPUB format. Maybe including this confirmation in the repo's README could be of help for those who stumble upon it.
And of course the AI salesforce was there pretending they take stuff seriously, maybe even believing it themselves. At least I don't believe that when the choice is maximizing profit or being a good person it will be the latter. Or at least I don't find a career path like that all too likely.
I wonder how many people read and heed the words of the Pope. I've seen letters like these for years now which sound good but I don't see any change in people after having read them.
I just messaged a few of my friends who I know are devoutly Catholic (and also happen to work in AI): 3 out of 4 are currently reading it this morning for their Memorial Day.
Really rather unfortunate. I wished religious people took their stuff seriously and not so often end up concluding that the bible wants them to harm people or something.
I guess it must be the same that make people think they must deliver freedom in form of bombs all the time.
Almost nobody.
Overview article: https://www.vaticannews.va/en/pope/news/2026-05/pope-leo-xiv...
The message here obviously comes from a good place. But it can't escape the trap baked into its own theology. Putting humanity one rung below God, with everything else below us is a trick that allowed semitic civilizations to make empires and economies go FOOM faster than the rest of the world. But it also makes things more dangerous when we start building systems smarter than we are.
What happens when the tool outgrows the toolmaker?
> But it also makes things more dangerous when we start building systems smarter than we are.
> What happens when the tool outgrows the toolmaker?
We've already built systems smarter than we are without much issue.Libraries and search engines for example. LLMs are just the next level of this.
AI tends to amplify the power of those who already possess economic resources, expertise and access to data
I’d bet the other way: You could have said exactly the same thing about computing 60 years ago, when IBM systems cost millions of dollars and filled whole rooms. And of course many people did.
Personal, commodity access to compute won, and won so thoroughly that it enabled this wave of scary compute centralization.
Centralized, scaled compute will always fill a purpose. But neither Microsoft, nor Facebook, nor OpenAI started out needing “Cloud Scale”.
The first one man unicorn startup will, I’m fairly certain, not be paying Anthropic per month or per token for the vast bulk of their matmul.
My secular take: "we want Star Trek, not Philip K. Dick, future".
AI must be “disarmed” in order to free it from the mentality of military, economic, and cognitive competition. “To disarm means discrediting the assumption that technical power automatically confers the right to govern,” he says. “To disarm does not mean rejecting technology, but preventing it from dominating humanity” (110). He devotes ample space to a critique of transhumanism and posthumanism, which interpret progress as the overcoming of human limits. Instead, limitations are not defects to be eliminated, but a constitutive dimension of the human person, because it is in fragility and finitude that relationship and openness to God and to others mature. He says we must remember that “humanity flourishes not despite limitations, but often through them” (118).
Pursuing technological innovation at the expense of eliminating human limitations, he says, would cause an anthropological regression. “Humanity—in all its grandeur and woundedness—must never be replaced or surpassed,” he says. Technology can alleviate humanity’s sufferings and open new possibilities, but it must not deny the essence of humanity, which is our “capacity for relationship and love” (126). In the face of AI, says the Pope, “the true alternative is not between enthusiasm and fear, but between two paths of development: a progress that serves individuals and peoples, or a progress that subjects them to the mentality of power” (129).
The paragraphs about making software transparent and collectively sound really similar to the open source ethos.
Cue Pieter Thiel explaining how this message of compassion is actually the word of the ant-christ while setting his software (maybe "built in Rust !) to all the earthly empires.
Interestingly, the Latin version of the encyclica is yet to be released at the moment
Modern encyclicals aren't written in Latin anymore. They're drafted in Italian and the title is the Latin translation of the incipit.
Indeed, the beginning of the Italian text is: "La magnifica umanità creata da Dio si trova oggi..." from which, Magnifica Humanitas.
For those who don't know why that's interesting: The tradional practice for creating these kinds of texts is to first write an editio typica in Latin. This Latin text is then the basis for translations into vernacular languages.
Yeah, I don't know how they expect the people of Latin America to read this.
Well because it’s Latin America, not the US, the vast majority of people are at least bilingual anyway :)
Although, it's called Latin America many countries speak variations of spanish and Brazil speak portuguese, just because those languages derived from latin and not because there the people speak it
That was the joke :(
If it helps, I got your joke
<3
Haha don't worry, next time I'll take care :'D
The net is too infused with dumb takes that the joke becomes indistinguishable to reality.
This is true, but we also can't go around adding /s to everything. Ah well, I guess it's a risk I have to take.
Pope Leo: Do Not Build The Torment Nexus
Techbros (as usual): how dare you suggest I'm a bad person for wanting to kill everyone in order to build the torment nexus? Don't you know how much money Jeff Bezos has?
As an atheist I have an obligation to finish reading it all (still going through, and taking notes, probably having to revisit), but I am not sure how many (christian) believers will feel the same.
As an atheist and even anti-theist I see no such obligation. What a strange thing to say
Priests will read it and then talk to their congregations about it on Sundays throughout the year, if not explicitly, then in how it shapes their homilies.
Some Catholic priests might do that, it’s up to the individuals.
From where does that obligation originate?
You mean Catholic believers, not Christian
Catholics are Christian.
In theory only and all Catholics recognize the authority of the pope. In practice it’s a mess as far as I understand, with a bunch of American catholic groups who rejected church reforms that happened during the 20th century, resulting in people calling themselves catholic who do not actually believe that the Catholic Church has authority over their religion.
Add to that the fact that the pope has a cultural influence that goes further than only the catholic audience (lots of Protestant see the pope as important even if that’s not something dictated by Protestantism, a bunch of not really religious people see him as a sort of spiritual leader, etc)
I’m sure a similar epistle long ago argued that Swordsmiths must ensure that their products are only used for justifiable self-defense.
I’d be thrilled if religion was only used to uplift people but that’s not going to happen either
Excited to read this. I really liked the note "Antiqua et Nova" from last year (still under Pope Francis). The autors showed a deep understanding of AI that many secular commentators lack. They developed the concept of integrated intelligence as opposed to the functional, reductivist view of intelligence that is prevalent in the AI community.
The em-dashes present within the writing made me pause and consider how much of this was written/exited by AI.
Quick browse through pre-AI works from John Paul II show em-dashes present.
Someone did an analysis and concluded that it appear to be at least partly (~10-15%) LLM-generated, or at least LLM-translated (see comments): https://www.lesswrong.com/posts/s58hDHX2GkFDbpGKD/linch-s-sh...
People need to stop acting like AI systems can detect AI. They can't. Pangram and similar are simply lying. There's no method to do what they claim and there never will be.
Pure speculation, but simply the presence of em-dashes may be a statement in itself.
One of the big problems I see currently is all the wild accusations being thrown around by seemingly half the internet that every little thing has been AI manipulated upon the tiniest suspicion. We will go mad tearing each other apart if we keep escalating this behavior.
Yes, some of it is blatantly obvious, but not to everyone-- so I think those casting aspersions need to really back up their claims with more than one or two bits of 'evidence'. I have been accused of using AI to write comments (which I have thus far never done), and I know I'm not the only one by a long shot. Such a waste of time and energy. Ignore it and move on if something smells off to you.
Also I am just so, so tired of the em-dash argument. Humans have been using it for a looong time. Let it go.
Now that's some em-dash passion!
My point was less about em-dashes and more stopping to consider how the vatican's workflow and editorial process has changed in wake of AI, and what, if any, impact that could have on the outputs.
AI is a tool, I have no problems with others using it to assist with writing as long as the original intent/argument remains.
for me the most important point in this is about how ai and tech in general concentrates power. if we want to build something good (in a moral sense) we need to put in work and make sure as many people as possible can use it with equal access.
this basically implies only open source models can be ethical but open source is not sufficient, you also need to make them give true information and avoid all kinds of harmful behavior. thats kind of a problem because if your weights are public even with a strict license a "bad" user can always fine tune it to remove any guardrails.
i think the solution for this is make sure the default behavior is aligned but let users turn on wild mode with zero censorship/refusals. that way everything is opt in, for example a parent can disable the mode for their children but a hacktivist or diy chemist can unlock everything.
as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
> as a self described good person i believe theres a lot more good people than bad people in the world (most are neutral) so if access to tech is equal the good side always wins. the problem here is again that access is not equal under capitalism. but thats a political thign not a tech one.
I believe there are mostly people who think they are good. And as the proverb goes the road to hell is paved with good intentions.
Has someone done an AI generated summary ;-)
Yes, but can the pipe draw a pelican riding a bike?
For a moment I pictured the Jesuits training their own LLM. If Arthur C. Clarke was still alive, we'd read a story like The Star, but with AI as the main plot device.
https://en.wikipedia.org/wiki/The_Star_%28Clarke_short_story...
Forgive me if this isn't what you meant, but a Jesuit has trained his own LLM:
https://www.magiscenter.com/magisai
From the "Core Features" tab: "Trusted sourcebase: answers are consistent with Catholic Church Teaching and the most contemporary scholarship in science, philosophy, history, scriptural exegesis, social science, and theology."
The Jesuit priest behind this is Fr. Robert J. Spitzer, S.J., Ph.D.
https://www.magiscenter.com/father-spitzer
I haven't used magisAI, but I've read a small to fair amount of Fr. Spitzer's writings, and also seen and heard some of his videos and podcasts (largely from his show Fr. Spitzer's Universe), and probably qualify as a big fan of his.
https://ondemand.ewtn.com/Home/Series/ondemand/video/en/fr-s...
https://www.ewtn.com/tv/shows/father-spitzers-universe
P.S. In case you are wondering about the glasses he wears or his appearance in photographs, he suffers near blindness due to retinitis pigmentosa:
https://www.magiscenter.com/blog/latest-news-on-fr-spitzers-... [2018]
EDIT: Formatting.
Ah, an opening to recommend The Sparrow by Mary Doria Russell, about a Jesuit going to space to speak with aliens. It won the Arthur C Clarke Award in 1998. I read it recently and it has stuck with me very strongly!
Now I’m thinking of “The Nine Billion Names of God” (which I actually read because someone here linked it!) and I’m a little nervous lol
That's a long read. I grew up Catholic, went through a pretty devout few years in my early adulthood, but ultimately I have decided that it is not for me. I send my kids to a Catholic school though (it is deeply tied to our culture), so I guess in that sense it is still worth my time reading it in full.
EDIT: Few paragraphs in, it is beautifully written.
I'm a non-practicing catholic, and an agnostic (or an atheist, depending on the mood). And yet I acknowledge that the Catholic church is a force to be reckoned with in the spiritual matters, and one of the few institutions to have had continuous influence on the material (or temporal) matters for centuries. Whether their brand of faith is rooted in your culture or not, these are words that deserve attention, I think.
(I'm not a Catholic though most people around me are.)
I have a similar perspective. Plus, I'll be frank: in the last few years, these occasional keynote publications from Vaticans are pretty much the most sane, deep, balanced and humane perspectives on AI anyone is writing. Reading this is a better use of one's time than reading the current batch of "tech thought leaders" articles or HBRs or Gartner magic square updates.
I'm a fedora wearing akshually agnostic and just too cool for bible school, but I never felt reading these "keynotes" wasted my time. They're not "just" humane but also very intelligent and direct. I would even go so far as to call them intellectually honest, and less religious that way than slogans like "you cannot stop progress, so just adapt", or something about toothpaste.
What caused you to fall away? Bad priest / diocese?
How many Catholics will put the encyclical to be summarized through an LLM?
If you asked all the LLM to find flaws in the arguments presented, and to come up with counter-arguments, I doubt many current models would be so bad as to come up with that, and even the ones that did would fold when asked "how is that even an argument?".
This was not an attempt at an argument, it was a sarcastic joke on how a lot of people are already dependent on AI, even on things that you weren't supposed to use LLMs for, but I guess I failed at it.
Fair enough, and kinda silly of me to assume so little of you.
Is that really true though, that so many people are "dependent" on "AI"? In what way? I'd say the only people who really depend on AI are those who want to make money off it, and that's only half sarcastic.
Would the people who now run it through an LLM (and who can read the output of an LLM but not the text itself?), have read it at all before? Would it not have, if anything, filtered down to them somehow, by them reading of it, or hearing of it in church etc?
It's almost a self analysis in my case, as when I saw the size of the text my mind immediately thought about feeding it into an LLM. Like, it's a text from the pope about AI, and my automatic reaction was to think about feeding it into an AI. What happened to me? And I think that happened to a lot of other people too.
ugh, read the room
No one who needs to read this will ever read it again.
Reading is a trained skill. Requiring years, even decades, of training. It too shall fall to AI.
This Dust short, "The Greatest Lie" seems timely
https://www.youtube.com/watch?v=gulzQIkwbJg
(at least the intro.. all in all I think it's a bit too long and some part of it kinda "cringe", but hey)
Why downvote this? The Holy See has an opinion on Artificial Intelligence? This is a fascinating document, and everybody should read it, and form their own opinions. The world currently seems to lack moral leadership, and who better to lead the cause of humanity than the Catholic Church?
> who better to lead the cause of humanity than the Catholic Church?
Oh no, is my sarcasm meter broken?
So the guys at Anthropic convinced the Pope to write a hype piece for them. Next-level grifting right there.
Why does the onus fall on the engineer for creating a better tool and not on the people who use that tool in evil ways?
We've been having the same argument since the dawn of mankind. AI is the new AR.
What kind of regulation is Leo the alignment expert proposing exactly? Occam's Razor says this is Anthropic trying to build a regulatory moat by leveraging some halo effect.
Maybe Leo should focus on finding a way to disconnect western society from their current cult-of-progress delusions? Could be a better use of the infallible man's pulpit?
From what you read, what do you think? All your questions are mentioned in the encíclica.
@dang Is this item getting a lot of negative votes? I've no way of knowing, other than seeing my kharma increasing only slightly after all the points the story collected.
Votes (particularly on submissions over comments) do not directly translate to karma. I'm not sure if it's documented anywhere what the algorithm is, but it's something proportionate to the logarithm of post votes becomes profile karma. It's similar with comments, but I believe (anecdata and observation) the positive effect of votes on karma is also logarithmic, but the negative effects of downvotes is linear; so if you have a highly controversial comment that sits at 1-2 points, you can net lose account karma.
Thanks! That makes sense. This was just genuine curiosity on how HN works, I couldn't care less about kharma.
The Pope is making a fundamental mistake here: His advisors told him that AI works and needs to be managed.
He does not address plagiarism, the fact that AI is mostly a surveillance and IP laundering tool, the fact that AI hasn't achieved much so far. You could say it has achieved nothing if compared to the whole history of human ingenuity, certainly not in CS.
He should have compared AI to the golden calf.
His criticism is lukewarm, does not address the criminal aspects and technological failures and is as such industry compliant. He can now say "I have tried" without harming the industry in the least.
This text is not what our current situation demands, but I hope that priests will augment and amplify it in their sermons and go a bit deeper.
Are people seriously turning towards their religions for a take on intellectual property rights?
Technically it is one of the ten commandments.
The part about not stealing? If yes that’s a tiny bit of a stretch :)