Impact

1.0 Testimonials
2.0 Case study
3.0 Survey results
4.0 Retention rate

On this page, I try to be like an open-minded yet rigorous researcher. However, I realize that the data I had to work with is substantially limited and I’m obviously biased, but at least it’s a start. I present and examine different lines of evidence around the cost-effective impact of my coaching. I erred in favor of comprehensiveness and neutrality while seeking appropriate consent from my coachees.

1.0 Testimonials

1.1 Abstract

Testimonials can be seen as mini case studies of the outcomes that certain people have experienced. The testimonials presented appear highly cost-effectively impactful – especially when considering the promise and impact of the people who experienced this effect. While it’s warranted to remain somewhat skeptical of both the content and selection of the testimonials, I think they’re useful proxies that can be trusted – especially for evaluations of “was it cost-effectively impactful for these people to get coaching from Sebastian?” or “is it promising for me to run a six-week experiment with Sebastian?”.

1.2 Results

The following are the six testimonials I collected from the people who worked with me in the past 1-6 months. Pictures, job descriptions, and number of sessions are included for context.

Weekly coaching sessions with Sebastian during 2021 have leveled me up as a leader and increased my focus and capabilities toward building safe AGI. He’s helped me clarify my long-term and short-term priorities and create systems to build new skills. These systems range from installing new habits to tracking systems to measure progress and hold myself accountable. It’s usually the highlight of my week in terms of sheer utility, and he’s also just a delightful and caring person!

Michael Andregg (Co-founder of Fathom Radiant )

Michael is co-founder of Fathom Radiant – a company building a new type of computer to enable safe machine intelligence who has investors like Jaan Tallinn and Jeff Bezos and has been advised by FHI’s GovAI.

After 30 sessions (Michael still works with me).

Sebastian has been a truly excellent coach. Our weekly conversations are rich and thoughtful. They always result in a good action plan, which steadily improve my productivity and, most importantly, my overall life trajectory.

Caroline Jeanmaire - Doctoral Researcher in Public Policy - AI Policy - Oxford University

After 8 sessions (Caroline continues to work with me).

Everybody should have a Seb. He really helps you take something chaotic and confusing and turn it into a manageable system where the next steps are clear. He’s got a great balance of helping you on the emotional side as well as the productivity side, and seeing how they relate to each other.

Kat Woods (Founder of Nonlinear)

Kat is the founder of Nonlinear, an organization focused on reducing existential and suffering risks by incubating cost-effective interventions.
After 5 sessions

Coaching helps me think of more habits, routines and thought patterns as mutable. Sebastian helped me to develop better systems for the structure of my workday, daily planning, and increasing time for deep work and improve my communication skills. Sebastian gives thoughtful feedback, asks the right questions, and helps to find the experiments that reduce or eliminate the core of the problem the situation.

Anonymous

After 12 sessions

I now trust myself and my abilities to do what’s aligned with my priorities and saying no to things that are ultimately unimportant. I went from procrastinating while feeling guilty and stressed, doing unimportant things optimized for improving my CV to setting realistic and motivating goals that are aligned with my values while enjoying more time off and peace of mind. I now also have solid habits around sleep and exercise and trust that I can implement new habits when needed. I would highly recommend coaching with Sebastian!

Mathias Bonde Chairperson of EA Denmark

After 15 sessions.

1.3 Should we trust testimonials as useful proxies for cost-effective impact in general?

Testimonials can be seen as mini case studies of the outcomes that certain people have experienced. While there are reasons to believe that testimonials overestimate the impact, there are also reasons to believe that they underestimate the impact. Overall, I think that some skepticism is warranted when interpreting testimonials but overall, they’re useful – especially if the people and the reported outcomes resonate with you. Finally, if the number of sessions (costs) are indicated that adds to the usefulness of the proxy.

Why it underestimates impact:

  • It’s a highly simplified representation of what happened and coaches typically spend 5-10 minutes writing it.
  • It can be hard as a coach to collect testimonials because making requests of others (especially if you know they’re busy) is uncomfortable so people have fewer testimonials than ideal.
  • Coaches aren’t aware of all of the change that actually happened.
  • Coachees may not want to attribute it to the coach because most humans like to give credit to themselves as opposed to others (it took me two years of deliberate work to strongly reduce this).

Why it overestimates impact:

  • It overestimates the impact because people want to be nice to the coach and only focus on the good sides. E.g., I haven’t encountered a testimonial that highlighted something negative.
  • The coach will likely cherry-pick the testimonials (I’m guilty of this in that in the beginning of my coaching practice I was more eager to seek out testimonials from people where I was confident that something really good happened as opposed to the two people that seemed to have gotten less out of it).

1.4 Should we trust these testimonials as useful proxies for cost-effective impact?

The testimonials presented appear cost-effectively impactful – especially when considering the promise and impact of the people who experienced this effect. While it’s warranted to remain somewhat skeptical of both the content and selection of the testimonials, I think they’re useful proxies that can be trusted – especially for evaluations of “was it cost-effectively impactful for these people to get coaching from Sebastian?” or “is it promising for me to run a six-week experiment with Sebastian?”.

2.0 Case study

The following is a case study of Mathias. I wrote up this case study based on survey results and my representation of what happened. Mathias validated and updated this six months after he stopped coaching with me (March 2022).
Mathias (15 sessions). Unfulfilled and under-utilized software developer and volunteer chairman of national EA group who transitioned into AI policy in the European Parliament and Charity Entrepreneurship

From (January 2021)

Life-time impact:

CS graduate with substantial un-utilized talent for policy/politics without much idea of what to do. Was working full-time as a software developer with low motivation and direction and organizing a national EA group in the evenings. Quite concerned with optimizing for roles that would look good on a resume (from a non-EA / common-sensical status perspective) and was applying for lobbyist jobs he wasn’t interested in.

Well-being:

Stressed and dissatisfied due to inability to prioritize, setting unrealistic expectations, and procrastinating. Unfulfilled due to a dissatisfying job. Poor dietary and exercise habits.

To (October 2021) (update March 2022 to fully capture the progress made and ripple effects)

Well-being:

More fulfilled as he is doing something that’s much closer aligned with future career goals. More satisfaction and having time for relaxation and fun due to increase in self-compassion, satisfying efforts during the day, and setting realistic expectations. Additionally, he now goes to the gym ~4-5 times a week and eats extremely healthily.

Improvement process:

Mathias spontaneously decided to set up a weekly coaching session where he writes things into a document, where he simulates Sebastian by asking the questions he imagined that Sebastian would ask.

Life-time impact:

Mathias formulated an ambitious decade vision of reducing X-risks via creating or contributing to institutions. Concretely, he went on to pursue AI governance in the European Parliament in a trainee position and is now actively looking to start a charity (participant at Charity Entrepreneurship Incubation program 2022). Frequently saying no to things that aren’t aligned with main priorities and doing deep work to focus on things that matter. A negative effect is that he is contributing much less to the national EA group but he found a full-time paid replacement.

3.0 Survey results

I haven’t managed to collect rigorous data from most of my coachees because i) they have an exceptionally high opportunity cost on their time and I haven’t felt comfortable asking more than twice, ii) my understanding of impact keeps evolving and there’s considerable interpersonal variation of what outcomes are the most important to facilitate which makes it difficult to standardize, and iii) doing this takes expertise (which I’m actively developing), time, and effort. However, I’m actively developing a survey that I can use every six weeks so hopefully this section will improve over the next months although I might also look into finding an independent and nuanced evaluator (let me know if you know good options).

The following are survey results from a pilot I ran within my first year of coaching with three people. Only two responded (they had gotten 16 and 5 sessions, respectively). Also, given that this is six months ago, I’ve probably improved by 40-70%. Please find a summary of the things that are appropriate to share with respect to the anonymity of the coachees. You can find a copy of the survey here.

Satisfaction

Both respondents reported being satisfied (4/5) with the coaching service.

Recommend to a friend or colleague

Both respondents reported being very likely (5/5) to recommend it to a friend or colleague.

The following are benefits experienced by both coachees (yes/no questions)

Decision making
Goal-setting, Prioritization, and (Purposeful) Productivity
Grit/Growth mindset/Agency
Behavioral literacy/Habit mastery
Deliberate practice/Learning
Emotional Intelligence/Self-mastery

The following are benefits reported by neither coachees (yes/no question)

Moral long-view/Transcendence
Truth-seeking/Scout mindset
Wisdom/Good Judgment

4.0 Retention rate

4.1 Executive summary

66% of the people who work with me continue beyond the first six-week experiment and go for at least 12 sessions (and as many as 50+) without any form of grant or subsidization. I take this as a strong indicator of cost-effective impact. While I don’t have formal data on how this compares to other coaches, my impression, based on conversations with other coaches and coachees, is that this is very promising.

4.2 Should we trust retention rate as a useful proxy for cost-effective impact?

Summary: I think that retention rate is a useful proxy for cost-effective impact as the opportunity costs (time and money) are too high for people to do this without it having substantial effects and that the counter-considerations aren’t that convincing.

Favorable considerations:

This far, I’ve only worked with people who have a fairly high opportunity cost on their time and they are selective about what they spend their time on.

They pay a significant sum for working with me and they’d much rather avoid that.

Retention could be seen as a revealed preference which weighs stronger than a stated preference (e.g., a positive testimonial) alone.

Counter-considerations:

People might continue just because they like me and our relationship (which was found in the psychotherapy literature) and have built an unhealthy dependency on me.

I don’t find this persuasive in my case because I cultivate a feedback culture that allows us to discuss “hard” or “awkward” things like this with radical honesty (which occasionally happens). Additionally, most are already highly independent individuals who have skeptical inclinations and express disagreements and wouldn’t do it if they didn’t find it valuable.

I don’t find this persuasive in my case because I cultivate a feedback culture that allows us to discuss “hard” or “awkward” things like this with radical honesty (which occasionally happens). Additionally, most are already highly independent individuals who have skeptical inclinations and express disagreements and wouldn’t do it if they didn’t find it valuable.

There might be something to this in that I only know two other coaches who do this (out of a total of nine) and this makes it less impressive compared to other coaches because maybe they’d have a much higher retention rate if they encouraged others. However, I hypothesize that the fact that I do encourage people is strongly correlated with what would make me a highly cost-effectively impactful coach: the ability to (and conviction in) facilitating significant (long-term) outcomes.

4.3 Why do people discontinue coaching with me?

I sought out feedback from the people who decided to discontinue coaching with me and here are the reasons mentioned (roughly in descending order of frequency/significance)
  • It’s too expensive to keep paying (e.g., because their financial situation changed).
  • Wanted lower frequency and shorter sessions than what I offer.
  • The counterfactual impact wasn’t high enough due to existing high-quality coach-like activities. Example: Weekly strategic and emotional conversations with partners and colleagues who have domain expertise in doing this.
  • Decent but not great personal fit/resonance.