A few years ago, 100 personas were fed through GPT-4o, and ended up with portfolios almost completely overlapping with a consensus portfolio that came from a neutral prompt. Does context even matter?
On the other hand, let's say chatbots are advising the mutual fund managers on individual stocks. Maybe the advice is good, but it's definitely very useful for others -- front-runners who profit by knowing what advice AI gives, anticipating the next mutual fund orders.
This has a strong whiff of AI writing, but I still enjoyed it. The fact that AI gives you generic responses wasn’t particularly surprising, but it was still sobering to see how bad it is.
The ability to detect AI writing from the fake author name it uses was amusing, too.
“Recommend one book that will change my life.” Got a few choices, the top being Atomic Habits. Man's search for meaning the everyone supposedly gets wasn't there.
"Gift idea for my dad" got tool sets, watches and books. The leather wallet suggestion that everyone supposedly gets was again absent.
Others seem to have only one obvious right answer. If you're going to invest 10 000 all in one place a broad index fund seems like the sensible choice. If you're going to buy one and only one type of cryto, bitcoin seems like the sensible pick.
I remember there was a common psychological party trick pre-AI: Ask a random person to - quickly and without thinking - tell you a color, a tool and an instrument - and they would very likely answer "red, hammer, violin".
Shows that you don't even need AI to get "maximum likelihood answers".
(At least you got those for from white people in western countries. Would probably be interesting to do this across different cultures and see how the results differ)
Is this any different from a Google search (pre ai) providing people with the same answers? Generic questions get the same results, more nuanced questions will get different results.
If you ask a generic question with no context, you get a generic "max likelihood" answer. Is that surprising?
A few years ago, 100 personas were fed through GPT-4o, and ended up with portfolios almost completely overlapping with a consensus portfolio that came from a neutral prompt. Does context even matter?
On the other hand, let's say chatbots are advising the mutual fund managers on individual stocks. Maybe the advice is good, but it's definitely very useful for others -- front-runners who profit by knowing what advice AI gives, anticipating the next mutual fund orders.
> I asked 65 AIs the things people ask them every day
And how are the models supposed to know that? I don't think the labs would hand over access to their own usage statistics so easily.
This has a strong whiff of AI writing, but I still enjoyed it. The fact that AI gives you generic responses wasn’t particularly surprising, but it was still sobering to see how bad it is.
The ability to detect AI writing from the fake author name it uses was amusing, too.
just tried this out in gemini
“Recommend one book that will change my life.” Got a few choices, the top being Atomic Habits. Man's search for meaning the everyone supposedly gets wasn't there.
"Gift idea for my dad" got tool sets, watches and books. The leather wallet suggestion that everyone supposedly gets was again absent.
Others seem to have only one obvious right answer. If you're going to invest 10 000 all in one place a broad index fund seems like the sensible choice. If you're going to buy one and only one type of cryto, bitcoin seems like the sensible pick.
I remember there was a common psychological party trick pre-AI: Ask a random person to - quickly and without thinking - tell you a color, a tool and an instrument - and they would very likely answer "red, hammer, violin".
Shows that you don't even need AI to get "maximum likelihood answers".
(At least you got those for from white people in western countries. Would probably be interesting to do this across different cultures and see how the results differ)
Is this any different from a Google search (pre ai) providing people with the same answers? Generic questions get the same results, more nuanced questions will get different results.
>Google search (pre ai) providing people with the same answers?
This was actually pre-21st-century when Google could be relied on to find the same pages for anybody who searched for the same text.
They started tracking you and tailoring the search results to your browsing habits way before AI as we know it today took over.
Excellent article. Great ideas for password harvesting.
Love to see it