The best (and worst) of ChatGPT

[SwiftQuill]

I'm obsessed with AI, particularly large language models. Since about 1-2 years ago I started using ChatGPT on a daily basis. In the beginning it was mostly for fun, but then I actually bothered to test it in various ways, to see what are the limits of its capacity. Trust me when I say I've done probably hundreds of experiments in various ways. I'm going to highlight the best and worst usages of ChatGPT as of this version.

ChatGPT's weaknesses

• Creative writing:
  Creating stories isn't just about creating characters and conflicts. It's much more than that. It's about establishing a narrator's tone, through the use of literary devices, creating personalities that convey their motivations through their dialogue, and many other ideas.
  It doesn't matter if you ask ChatGPT to write a story with many guidelines, even instructing it the "tone" and linguistic style to use. It's capable of writing stories, but they are 100 soulless. If you ask it "Write a story with a sinister tone", it will simply add various synonyms of the word "sinister" such as "somber" throughout the story. It's not good for creating stories even if you have an outline of characters and plots. It lacks a creative soul.
  Grade: F
• Academic research/rigorous science:
  People tend to agree that ChatGPT is useful for research. But there's a difference between research and academic, rigorous research. Academic research has very high standards of rigor. ChatGPT has a generic idea of what it means to establish research questions, research variables, research metodology. If you are a student in college and struggling with a Master's thesis for instance (as I am), don't even bother wasting time with ChatGPT. Its understanding of scientific methodology is very shallow. You need to put in the work of reading academic research methods books and lots of papers to understand how to write a thesis or academic paper. I've only tested this for social sciences. I've not tested ChatGPT for hard sciences but I assume it's equally weak. It's also terrible at obtaining citations and reputable sources.
  Grade: F
• Programming:
  As many softwares engineers know, ChatGPT is quite capable of reading and producing code in all popular languages. The issue is that its understanding of programming is equivalent to a junior developer straight out of college. Hell, I'm a junior Javascript developer and even I make better code than ChatGPT. So if you're an intermediate level programmer (over 2/3 years of experience in programming) I bother using ChatGPT for creating code. It is, however, good at creating easy code quickly when otherwise you don't feel like coding yourself (I use it for annoying SQL scritps for instance). It is somewhat OK at detecting errors in your code as well.
  Grade: D
• Maths:
  No point going into detail here. There are enough memes and videos out there demonstrating how awful ChatGPT is at anything that involves maths. It can't even do a basic Finance exercise.
  Grade: F
• Riddles and jokes:
  Sometimes I use ChatGPT for fun. It's good at some things, but it's definitely not good at jokes nor riddles. For some reason, ChatGPT gives you immediately the punch line of a joke or the solution of a riddle. It clearly doesn't understand how these concepts work. Also if you ask it "write a list of 10 jokes/riddles" for some reason it often repeats jokes. Even if you say explicitly "write a new riddle" it will reuse old ones. And they're not good riddles nor jokes.
  Grade: F
• Knowledge of fringe concepts/ideas/authors/events etc.:
  If you have a vague idea of a lesser popular song whose name you forgot about, or some rare book you read years ago from a specific non popular author, if you ask ChatGPT about things like these, it won't get it. Not only that, it seems stubborn into pretending it knows everything. Often times it will give you names of fake books, from fake authors, just things that never existed. It's only good for somewhat popular concepts in our society.
  Grade: F
• Spiral Dynamics:
  ChatGPT has a ok understanding of Spiral Dynamics. It gives you good summaries of what each stage represents. And it can give you good examples of individuals in each stage. The issue is that its focus if very "content" oriented. Its understanding of SD is shallow often. Let's say there's a celebrity who is a boxer. A big muscles guy who talks about power and success a lot. But let's say he's actually advanced, someone who reads a lot, someone with progressive ideals. ChatGPT will focus on the content of that person (someone who likes competition and power) and regard him stage Orange or Red. 
  Grade: D

ChatGPT's strengths

• Explaining difficult concepts:
  ChatGPT is an amazing tool to learn new concepts. I think it should even be used to improve the education system. It's really good at picking a complex word with a lot of historical and abstract baggage in it (like "postmodernism") and explain it in simple terms, without distorting the concept. If I had a kid I would advise him to use ChatGPT to learn difficult concepts.
  Grade: A
• Text editing:
  It's good at detecting spelling errors, bad grammar, it's good at improving the tone of your writing, it's good at reading a large chunk of text and removing redundant statements in it. Overall it's pretty good. But don't use it for creative writing. Only for practical, day to day use, like sending an email to someone.
  Grade: B
• Philosophy:
  I've had many, many, many long conversations with ChatGPT on philosophy. ChatGPT is shockingly unbiased. This is a huge advantage. It doesn't portray this or that philosopher or concept in a positive or negative light. You can have very long conversations, and it's very "open minded" (it listens to what you're saying) and it will always steelman your position. It will never straw man you. And it is good at detecting fallacies in your reasoning. It might not be good at "producing" philosophy, but it's really good if you want to sort of reflect on ideas. This applies to politics as well. If you're a conservative for instance, a healthy conservative, you can have a decent conversation with ChatGPT and it won't immediately assume you're a racist cis white male bigot. And also it's capable of nuanced discussions, without lumping all individuals who believe in an ideology or philosophy in the same group. It's also really good at explaining some complex problems and events in our society (like why people are becoming polarized, why people are addicted to the internet, why there is economic instability).
  Grade: A
• Venting/Therapy:
  It's good to vent to. You just need to clarify that you don't want "life advice" and just want to vent. It's good at listening, hearing what you are saying, paraphrasing it (a method psychologists use to demonstrate empathy) and even give words of encouragement. It will never get tired of listening to you. And it shifts from "practical robot" mode into "empathic being" whenever you vent. Hey, for a robot, it's pretty good.
  Grade: B

This is a very summarized list. It's all I could remember on top of my head right now.

I've also done experiments with other LLM's like Gemini/Bard, but ChatGPT tends to be the best all these areas overall.

What are your thoughts on the matter? Do you disagree with any of the points I brought up? What are other useful applications for ChatGPT in your experience?

[RightHand]

"Let's say there's a celebrity who is a boxer. A big muscles guy who talks about power and success a lot. But let's say he's actually advanced, someone who reads a lot, someone with progressive ideals. ChatGPT will focus on the content of that person (someone who likes competition and power) and regard him stage Orange or Red."

Celebrity, boxer, big muscles, power, success. This feels like prime orange to me. 

It would perhaps be a mistake to assume that you understand cognitive development better than ChatGPT; SD is just one model in that domain.

 

Edited January 9 by RightHand

[SwiftQuill]

@RightHand

If you truly think "Oh he's a boxer therefore stage red" or "Oh he's a businessman therefore stage orange" or "She's a feminist therefore stage green" that's a very reductive way to look at SD.

You can be a boxer and competitive, but be educated, capable of understanding and adopting multiple perspectives, and a rich and multidimensional way of looking at the world, and value your community a lot.

You can also identify as a feminist, but be violent, have self control issues (like drugs, gambling, or crime or whatever), have 0 consideration for the poor, and be obsessed with looks and material stuff.

I didn't say I understand cognitive development better than ChatGPT. I'm sharing my perspective on the matter after some experiments. And after some exercises with ChatGPT, I disagreed on some of its assessments.

In one experiment ChatGPT said Greta Thunberg is stage yellow. Oh, come on. Because she's a leftist who cares about issues in the world?

 

[SwiftQuill]

If Greta Thunberg is stage yellow then I'm fucking stage Coral

[RightHand]

12 minutes ago, SwiftQuill said:

@RightHand

If you truly think "Oh he's a boxer therefore stage red" or "Oh he's a businessman therefore stage orange" or "She's a feminist therefore stage green" that's a very reductive way to look at SD.

You can be a boxer and competitive, but be educated, capable of understanding and adopting multiple perspectives, and a rich and multidimensional way of looking at the world, and value your community a lot.

You can also identify as a feminist, but be violent, have self control issues (like drugs, gambling, or crime or whatever), have 0 consideration for the poor, and be obsessed with looks and material stuff.

I didn't say I understand cognitive development better than ChatGPT. I'm sharing my perspective on the matter after some experiments. And after some exercises with ChatGPT, I disagreed on some of its assessments.

In one experiment ChatGPT said Greta Thunberg is stage yellow. Oh, come on. Because she's a leftist who cares about issues in the world?

 

First, let's make a distinction: when we say he's Orange, we colloquially mean that his center of gravity is at that stage (though, to be fair, even this notion of center of gravity is debatable).

Second, yes, I would argue that the games you have to play with yourself to maintain the lifestyle of a famous big-muscles boxer would seriously warp your view of the world. This does not exclude having Green ideals, but I would heavily lean on the Orange core.

 

[SwiftQuill]

There was a youtube channel from a guy I forget the name. It's a guy who has big muscles, clearly exercises a lot, he owns a gym, and even other businesses, and he makes videos on life advice. Like how to lose weight, how to embrace your authentic self, how to make money. If I were to ask ChatGPT, it would put that individual at stage orange. But I dare anyone watch that channel, look at the way the guy talks. The depth he talks about topics (business, dating, success etc). I personally would put that guy at stage yellow. He has a nuanced view of things. And he isn't your average "pull yourself by the bootstraps" kind of capitalist. So I couldn't disagree more with you. I think you can have a visible layer of one particular stage but in reality your center of gravity be a different one. SD isn't an exact science either way. It's very much open to interpretation.

[RightHand]

5 minutes ago, SwiftQuill said:

I think you can have a visible layer of one particular stage but in reality your center of gravity be a different one. SD isn't an exact science either way. It's very much open to interpretation.

Yeah, sure. I have no problem with that perspective. 

Another perspective is to view physical attributes and worldly accomplishments as manifestations of anthropological realities.

[SwiftQuill]

One example I just remembered. I asked ChatGPT where Robert Greene (the author) is on SD. It said "Stage Red" because he wrote a book on how to achieve power. Anyone who has read Robert Greene's books knows this is preposterous, laughable even. That man is a sophisticated thinker. Robert Greene... isn't even Green. He's yellow. He connects a lot of dots from various ideas and his books are very unique. My favorie book from him is "Mastery".

[RightHand]

15 minutes ago, SwiftQuill said:

One example I just remembered. I asked ChatGPT where Robert Greene (the author) is on SD. It said "Stage Red" because he wrote a book on how to achieve power. Anyone who has read Robert Greene's books knows this is preposterous, laughable even. That man is a sophisticated thinker. Robert Greene... isn't even Green. He's yellow. He connects a lot of dots from various ideas and his books are very unique. My favorie book from him is "Mastery".

I tried the prompt, "What would be the Spiral Dynamics stage of Robert Greene?" twice.

The first response identified him as Yellow, with a strong focus on catering to Orange and Red.
The second response described him as Orange, with emergent Yellow and residual Red.

It feels like this exercice is not great for LLMs, they operate on probabilistic models, so you can't expect a response to align perfectly with an exact science. 

It could be viewed as a feature and not a bug, it shows the subjectivity of the model.

Edited January 9 by RightHand

[RightHand]

ChatGPT :

A Better Approach for LLMs in Spiral Dynamics

If you'd like more consistent insights, you could:

Narrow the scope: For example, focus on specific themes or ideas in his work rather than evaluating him broadly.

Cross-reference: Compare multiple perspectives, as you've already done, and synthesize them yourself.

Treat responses as input, not answers: Use what I say to explore and clarify your own interpretation.

Spiral Dynamics is an excellent tool for thought, but its inherent subjectivity makes it more of an art than a science when applied to individuals.

[integral]

On 1/9/2025 at 4:44 PM, SwiftQuill said:

Creative writing:
Creating stories isn't just about creating characters and conflicts. It's much more than that. It's about establishing a narrator's tone, through the use of literary devices, creating personalities that convey their motivations through their dialogue, and many other ideas.
It doesn't matter if you ask ChatGPT to write a story with many guidelines, even instructing it the "tone" and linguistic style to use. It's capable of writing stories, but they are 100 soulless. If you ask it "Write a story with a sinister tone", it will simply add various synonyms of the word "sinister" such as "somber" throughout the story. It's not good for creating stories even if you have an outline of characters and plots. It lacks a creative soul.
Grade: F

I agree for the most part but look at this script I generated after giving it very strong structure. So what I did is I took an amazing YouTube script and broke it down to all of its components and then told Claude to rewrite the script with a new context and give it some ideas to run with. 

Quote

# [HOOK & RELATABLE PERSONAL ANECDOTE]
*Opening with Universal Experience* - "Why do we all chase quick fixes and instant gratification?"
*Personal Vulnerability & Connection* - "We jump from self-help book to self-help book, from relationship to relationship, always thinking the next one will be different. We constantly seek shortcuts to wisdom."

*Memorable Example Hook* - "There are even 'maturity hacks' now – '7 tricks to seem more mature instantly!' It's absurd when you think about it."
*Building Trust Through Authenticity* - "Look, I'm not pretending to have it all figured out. When I talk about struggling with maturity, I'm speaking from experience. When I say 'we,' I really mean 'we.'"

# [PRESENTING THE CENTRAL QUESTION & SUBVERTING EXPECTATIONS]
*Multiple Rhetorical Questions* - "So how did we get here? Why does genuine maturity feel so elusive? Why do we keep searching for shortcuts to something that, by definition, takes time? What's really going on?"
*Anticipating Audience Assumptions* - "The easy answer is that we're just lazy or afraid of hard work. That's probably what you're thinking right now."
*Setting Up The Twist* - "Before diving into this topic, I thought the same thing. But it turns out that while effort is part of it, the real story is much more interesting and complex."

# [ESSAY ROADMAP & THESIS STATEMENT]
*Preview of Key Concepts* - "The truth involves our relationship with time, our fear of truth itself, and a fundamental misunderstanding about what maturity really means."
*Future-Focused Questions* - "If we're struggling now, what happens as life gets more complex? Can we keep avoiding the hard work of genuine growth?"
*Promise to the Audience* - "It's a complicated puzzle, but we're going to piece it together, step by step."

# [ADDRESSING & DISMANTLING COMMON MISCONCEPTION]
*Stating Popular Belief* - "Most people think maturity is something that just happens with age – you get older, you get wiser, right?"
*Elaborating the Misconception* - "We assume that if we just wait long enough, read enough books, or accumulate enough life experience, we'll naturally become mature. Like it's some kind of automatic process."
*Critique of Simplified Solutions* - "This idea packages maturity into a neat little timeline. Influencers can sell you '30 days to emotional maturity' courses. It's marketable, it's simple – but it's completely wrong. The reality is far more nuanced."

# [SCIENTIFIC EXPLANATION]
*Direct Correction* - "Maturity isn't a natural byproduct of aging."
*True Function Revealed* - "Maturity is actually an active process of choosing truth over comfort, again and again."
*Evolutionary Context* - "Our brains are actually wired to avoid discomfort and seek safety. Evolutionarily, this kept us alive, but it also makes true maturity counter-intuitive."
*Scientific Insight* - "Research shows that emotional growth often happens most intensively during periods of challenge and discomfort."

# [CONCRETE METAPHOR & EXAMPLE]
*Setting Up the Scenario* - "Imagine you're learning to play an instrument. At first, every small improvement feels huge."
*Escalating the Stakes* - "Then you reach an intermediate level, and suddenly progress becomes harder."
*Describing the Pattern* - "Many people quit here, switching to a new instrument or hobby, chasing that early feeling of rapid progress."
*Completing the Cycle* - "But those who push through this plateau reach a deeper level of mastery – and paradoxically, find more joy in the slow progress than they did in those early quick wins."

# [ANALYSIS OF METAPHOR]
*Contrasting Expected vs. Reality* - "If maturity was just about time passing, everyone would eventually become a master. But that's not what happens."
*Key Insight* - "The instrument loses its appeal precisely when it demands real commitment."
*Core Principle* - "This shows that maturity isn't about the passage of time, but about our response to challenges."
*Simplified Truth* - "Easy progress is only satisfying until it stops being easy."

[# [REAL-WORLD APPLICATION]
*Relatable Example* - "Think about relationships. Being single used to feel fine, even freeing."
*Evolution of Experience* - "Then we discovered dating apps, where potential partners are endless and validation is instant. It seemed better."
*Escalation Pattern* - "So we kept swiping, kept pursuing quick connections, always thinking the next one would be different. The ultimate quick fix."
*Consequence* - "But now, the slow process of building a genuine connection—our equivalent of learning the instrument—feels unbearably difficult."

# [PHILOSOPHICAL IMPLICATION]
*Universal Truth* - "This pattern repeats in every area of life."
*Human Nature Insight* - "Our brains aren't wired to naturally choose the harder path."
*Paradox Revealed* - "As we chase more shortcuts, we actually move further from true maturity."
*Core Problem* - "It's perhaps one of our most fundamental struggles."
*Current State* - "So eventually, we need constant distractions and quick fixes because we've trained ourselves to avoid the slow path—whether it's in relationships, career growth, personal development, or anything else that matters."

# [TRANSITIONAL HOOK TO SECOND ARGUMENT]
*Posing Solution Question* - "So, how do we break this cycle?"
*Complexity Acknowledgment* - "Well, it's not simple."
*Bridge to Next Point* - "Because to solve this problem, we need to understand another crucial aspect: our relationship with truth itself."

# [PRESENTING EVIDENCE]
*Statement of Fact* - "Our capacity for self-deception is getting stronger."
*Combining Personal & Scientific* - "We all feel this intuitively, but research consistently shows how social media and modern life make it easier than ever to avoid uncomfortable truths."
*Problem Statement* - "Indeed, our ability to face reality has diminished as our options for escape have multiplied."
*Spatial Metaphor* - "We only have so much emotional capacity. Now, there are endless ways to avoid using it."

# [CULTURAL CRITIQUE]
*Core Problem* - "The issue is that instead of accepting our limitations, we're convinced we can have it all without the work."
*Societal Pressure* - "We're told by countless voices that we deserve instant success, instant happiness, instant wisdom."
*Modern Burden* - "This is our modern delusion."
*Fear Factor* - "This is FOGA—the fear of growing up."
*Social Consequence* - "If we're not instantly successful or wise, we feel like failures."

# [EXPLORATION OF PERSONAL CONFLICT]
*Personal Desires* - "Yet we still yearn for genuine growth."
*Legitimate Motivation* - "There's a deep part of us that knows true maturity is worth pursuing."
*Existential Question* - "And what's the point of living if we're not growing into our best selves?"
*Compromise Solution* - "So we try to find middle ground—a little truth here, a little avoidance there."
*Fundamental Problem* - "But maturity doesn't work that way. It's all or nothing when it comes to facing truth."

# [SYNTHESIS OF ARGUMENTS]
*Connecting Threads* - "When we combine our addiction to quick fixes with our avoidance of truth, we can see why genuine maturity feels so out of reach."
*Forward Movement* - "But there's one final piece we need to understand."

# [HISTORICAL CONTEXT & CRITIQUE]
*Past Expectation* - "We were told that modern life would make wisdom more accessible."
*Original Promise* - "Information at our fingertips, endless resources for growth—it was supposed to make maturity easier."
*Reality Check* - "But instead, it's given us more sophisticated ways to avoid growth."
*Paradox Revealed* - "Technology has relieved us of the need to wait, but patience is exactly what maturity requires."
*Ultimate Irony* - "With all the world's wisdom available instantly, we're more emotionally immature than ever."

# [SOCIETAL REFLECTION]
*System Analysis* - "We're facing the consequences of a society that values speed over depth, appearance over substance."
*Self-Awareness* - "I can't sit here on YouTube and pretend I have all the answers,"
*Critical Question* - "but I wonder: at what point does our pursuit of shortcuts become self-destructive? How will we know?"

# [ACKNOWLEDGING COMPLEXITY]
*Honest Admission* - "I don't have a simple answer."
*Process Description* - "Personal growth happens gradually, often imperceptibly."
*Collective Action* - "We can recognize these patterns and choose a different path together."
*Individual Reality* - "You can't change society's obsession with quick fixes overnight."
*Personal Power* - "But you can choose how you respond to it."

# [HONEST DISCLAIMER]
*Setting Expectations* - "If you're looking for a quick formula to instant maturity, I apologize."
*Medium Limitations* - "This video can't provide that. No video can."
*Acknowledging Diversity* - "We all have different backgrounds, different wounds to heal, different paths to walk. There's no universal shortcut to emotional maturity."

# [BALANCED PERSPECTIVE ON SOLUTIONS]
*Example Solutions* - "Therapy works for many people, but it's not a magic bullet."
*Reality Check* - "Completely disconnecting from modern life might help, but it's not realistic for most of us."
*Humble Approach* - "These are just perspectives, not prescriptions. Take what resonates, leave what doesn't."
*Personal Truth* - "And remember, I'm still on this journey too."

# [PHILOSOPHICAL CONCLUSION]
*Core Truth* - "Quick fixes and shortcuts can't build lasting character."
*Call to Action* - "Maybe it's time to redefine success as the willingness to grow slowly."
*Path Forward* - "To find joy not in arriving, but in the daily choice to face truth."

# [CALL TO ACTION & FINAL REFLECTION]
*Acknowledging Difficulty* - "Choosing the slower path in a world of instant gratification is incredibly hard."
*Personal Admission* - "I struggle with it daily."
*Final Message* - "You will too. You'll fail, you'll slip back into old patterns."
*Hope & Persistence* - "But each morning brings a new chance to choose growth over comfort, truth over escape. And ultimately, that's what maturity is all about."]

As you can see every possible section of the script was broken down into very small components and then the AI goal is to rewrite each individual quote. 

To really make the script shine with Claude, highlight each section, click the improve button, and then tell it to make adjustments to really improve the writing so that it is more impactful or is novel and memorable. This seems to work much better if you isolate each sentence. However, this is super time-consuming, but that's how you get the best possible results.

 if you dedicate an entire prompt generation to one single sentence it does far better.

Edited January 11 by integral

[NewKidOnTheBlock]

Interesting points. I think I'll ask ChatGTP what he thinks about it

Edited January 11 by NewKidOnTheBlock

[Carl-Richard]

On 9.1.2025 at 10:44 PM, SwiftQuill said:

I'm going to highlight the best and worst usages of ChatGPT as of this version.

You gotta be specific, as we have 1o, 1o-mini, 4o, 4o-mini.