Why I Switched to SuperMemo After Using Anki for 5 Years, With Over 50k Cards and 420k Total Reviews

Introduction

My story might be worth reading because I:

- Have been using Anki since Sept 2013 (more than 5 years)

- Tried SuperMemo in March 2018 (7 months ago)

- Abandoned Anki and switched to SuperMemo completely

My Story with Anki

I remember reading the book Make It Stick and Anki was the first Spaced-repetition Software I encountered that directly applies the learning principles described: spaced repetition and retrieval practice. I created my Anki account on Sept 5, 2013, so more than 5 years ago. Ever since I have become a diehard fan of Anki:

My Anki Statistics

Three periods with major changes to the same profile:

- During high school years

- College entrance exam

- Last for language learning (I was trying to learn Korean)

Total Review: 420,003

Total Review Time: 1,538 hours

Total cards: 21695 + 18582 + 10052 = 50329, so over 50,000 cards

PS: I just realized I even have the collection.apkg from Sept 14, 2013. Remember to back up my fellow Anki users!

Compiling the statistics for this post made me realize how much time and effort I had put into Anki. There were countless hours staring at the screen, making new cards and doing my reps. I had poured my soul and heart into Anki.

My Love for Anki

- Bought the $25 iOS Anki just to show support (I don’t even own an iPhone)

- Donated twice before Damien stopped accepting donations: one on Sept 19, 2013 and another May 16, 2014. I’m even surprised that I could see the potential and benefits of SRS just 2 weeks after my first encounter.

- Started learning Python because Anki is written in Python and I wanted to create add-on.

I mention all these stats and evidence to show that I love Anki and never ever will I hate it just because I’d switched. Anki has saved my academic life.

My Failed Attempt at Switching to SuperMemo

I tried SuperMemo back in 2016. When I was searching about it, everything I searched online was about switching FROM SuperMemo to Anki, not the other way around. I was constantly questioning myself why I was trying to abandon Anki.

Regardless, I tried SuperMemo 15. Since I was still very pro-Anki, I was trying it with a mindset that’s very unfavorable to SuperMemo. When I first opened SuperMemo,

Damn, the interface looks terrible.

There are many confusing buttons and options and symbols. I went through the ABC of SuperMemo, a 5-minute tutorial on the basics. Then the frustration cycle started:

1. Trying to do something, like changing the font, template, inserting images/mp3

2. Couldn’t figure it out

3. Searched online

4. Only super-lengthy articles from SuperMemo’s official sites. I could never find the exact solution to my problem, only related information and hoping to derive a solution. No one on the Internet seemed to be using SuperMemo. At the same time, I was always thinking: “Anki is better in this; Anki could do this better.”

After few such rounds, I quitted. I don’t think I lasted a few days before deleting SuperMemo. In retrospection, I didn’t use it long enough to see the points of Incremental Reading. My complaints with the UI and its complexity were means to ease the cognitive dissonance, excuses to make myself less likely to switch. It was not easy trying to switch:

Cognitive Dissonance and Confirmation Bias

1. Cognitive Dissonance

Switching meant admitting that I was wrong all along. After all, I’d been using Anki for 3 years already and switching means that there was a better tool out there that I wasn’t using. Such cognitive dissonance was hard to overcome. I was resisting, desperately clinging onto what I believed to be true and right (Anki is the best).

For example, while researching I came across What spaced repetition algorithm does Anki use? and Algorithm SM-17 vs. older SuperMemos. I’d totally agree what Damien said and quickly dismissed that supermemopedia page:

Yes. Damien’s comment is totally legit. “All those detailed considerations should not obscure the fact that the Algorithm SM-2 works great in practice, and algorithmic fine-tuning is important more for theoretical than for practical reasons.” So Anki it is!

Essentially, SuperMemo and Anki are mutually exclusive: agreeing with one side means disagreeing to the other. Since I was pro-Anki, I quickly suppressed the doubt of a better alternative.

2. Confirmation Bias

Whenever I searched on Reddit, everyone was so Pro-Anki (It didn’t help that I’d also search inside r/Anki). Everyone was saying how Anki was better than SuperMemo and asking how to switch from SuperMemo to Anki.

Googling Anki vs. SuperMemo and you’d see what I felt like was two sides: the world of Anki users vs. Woz himself

The popularity of Anki outmatched SuperMemo:

- There is a learning-Japanese forum dedicated to using Anki to learn Japanese

- Medical students sharing Anki decks

- My favorite r/Anki.

On the other side, there’s Woz and only Woz. He has been updating his wiki and help site

It’s 100% without doubt that his words about SRS and SuperMemo are biased. He is the creator of spaced repetition algorithms and SuperMemo after all. I took extra grain of salt when reading his words. Given his position, I just didn’t think his words were trustworthy.

There were some blog posts online about Anki vs. SuperMemo but they were not very helpful and in-depth enough for me; there was a Google forum but it’s mostly dead: no active discussions, active members, no nothing.

My point is, it seemed to me that:

- The world was using Anki. SRS = Anki

- Everyone was against SuperMemo and SuperMemo users were switching to Anki

- Only the creator was defending his software

That was it: I deleted SuperMemo and continued using Anki. Not until 2 years later did I try to give it another shot.

Why I Switched to SuperMemo

There were two nagging reasons:

1. Burning out in Anki

2. Believing in the superiority of the SM-17 algorithm

However, my burnt-out experience and the algorithm were not enough to make me switch. These two reasons were the final push:

1. Frustration with Transfer of Knowledge in Anki

2. Realizing the benefits of Incremental Reading

My burnout in Anki was due to:

1. High review count

2. Frustrated with the Anki settings

The high review count made me tweak the Anki setting. Yes, tweaking it lowered the number of reviews, but it also made me paranoid about messing up the algorithm and lowering efficiency and effectiveness. In order to justify my new setting, I kept researching and this only led to more frustration.

1. Burning Out In Anki

1. High review count

The massive number of reviews was especially apparent when I started learning Korean. If I didn’t finish my reps that day, the review count would keep accumulating and before I know, 100 cards would have turned into 800 cards.

The picture is from Nov 22, 2016: I studied 1163 cards in 255 minutes (more than 4 hours). If memory serves, I should have spent way more than 4 hours that day doing my reps.

There were many days when I had to review over 500 cards for a deck and I had multiple such decks. I remember being faithfully drilling vocabulary for hours every day, day in and day out, going through all the reviews. After a few days, I would see progress of reduced reviews from 500 to 300 to 200…

Then if I didn’t do my reps for a few days, the review would SHOOT to over 1000 cards. It was either due to laziness (“See? The review count has subsided. Let me take a break.") or stuff that happened in life. Then I had to go through the reviews again. Every day, I fired up the Desktop Anki, going through the reviews; do my reps in Ankidroid whenever I had time here and there. Usually, the review count was so high that I couldn’t add more new cards.

After doing this for a month, I was burnt out. I wanted to stop learning altogether, but I also didn’t want to give up learning. This always went on in my head:

“Should I keep doing my reps? But there are so many cards left for review… But if I don’t review and let the review builds up… I will suffer even more the next day…”

Yes, you may argue that was not the right way to learn a foreign language. Regardless, this experience led me to look into the Anki settings.

2. Frustrated with the Anki Settings

Due to the high review counts from drilling vocabulary, I started reading about the literature on memory and looking into the Anki setting and the algorithm. From Desirable difficulty Vs. Anki setting for vocabulary acquisition:

Therefore, too easy retrieval should be avoided. […] Or if I did fail, how will I know that it’s for better subsequent learning, or my setting is too harsh that I actually completely forgot the material?”

How to balance that fine line between desirable difficulty and Anki setting? How to tell if it’s too much, too difficult for me?

What I fear is that MORE reviews actually HURTS my retention rate.

From My 5-month findings after messing with the Anki setting:

“It seems counter-intuitive that with such large intervals, how is one going to memorize efficiently?”

“I can’t believe I’ve been using the default setting all this time, which throws the card back to interval of 1 day when I pressed Again.”

I was quite serious about tweaking the settings. I mostly followed the setting of a Japanese-learning “guru” who would cite research and back up his claims with studies. To me, he was more trustworthy than other users' personal preferences.

My 5-month findings were obviously, all trash and noise. I couldn’t draw any meaningful conclusion: there were just too many variables. In retrospect, I didn’t even know how to properly design my “experiment”; it’d already failed before I even started. But my experiment did highlight that no one cares much about algorithm optimization. I couldn’t find (good) answers online so I had to try to do my own experiment. It seemed to me that most don’t care about the significance of the algorithm has on long-term retention and repetition workload.

My investigation into the settings and algorithm optimization only led to more frustration and questions. There are a lot of parameters and settings in Anki: Steps, Graduating Interval, Easy Interval, Starting Ease, Easy bonus, Interval Modifier, Steps and New Interval for lapses.

It’s not difficult understanding what each parameter means. The real question is how each parameter affects each other. Different parameters are so interconnected that changing one parameter could have a huge influence on review counts.

In Anki, maybe the combination of decreasing the Starting Ease and Easy Bonus WHILE increasing the Steps and New Interval for lapses works better. Or every 10% increase of Starting Ease has to be matched with 30% decrease of the Easy Bonus.

I simply did not know how to change them. I still don’t. Since I wouldn’t know how my changes had affected my long-term retention, I got paranoid about undermining the algorithm’s efficiency and wasting my time.

You may argue, “Hey, you can check the statistics and figure it out!” But I don’t think the statistics could give me a clear answer: I don’t have any criteria to determine the effectiveness the “modified algorithm” on my subsequent learning. I was aware that I could tweak the Interval Modifier according to this formula, log(90%) / log(85%) however, the expected result is only valid under the condition that you follow the default setting and only change the Interval Modifier. If you’ve also changed other parameters, that formula doesn’t take into account of how other parameters affect the algorithm.

For example, you’ve discovered the brutality of the New Interval of lapses in Anki (more on this later). How do you proposed to solve it? Do you change the New Interval of lapses to 30%, 50% or 70%? Based on what criteria, reason(s), evidence? For example, “Have I increased the Interval Modifier too much that too many repetitions are hurting my retention?"

1. More ≠ Better And Higher Retention

The more I review the same card, the more I might be actually hurting my long-term retention:

If an item had not already made a transition to the learned state, but was available in short-term memory (owing to recency), studying that item had no effect on its learning.

It is important to realize not only that effortful retrieval can enhance learning, but also that the converse is true as well: Trivially easy retrievals appear to result in essentially no learning._ On the Symbiosis of Remembering, Forgetting, and Learning

Giving several tests under conditions that are too easy undermines the positive effects of testing. In the 0-0-0 condition subjects were required to recall items three times under conditions in which they were essentially always correct. However, these three (easy) retrievals led to later retention that was even worse than a single test given under more difficult conditions (the 5 conditions at both delays). Intricacies of Spaced Retrieval: a Resolution

Manipulations such as keeping conditions constant and predictable and massing trials on a given task often appear to enhance the rate of learning during instruction or training, but then typically fail to support long-term retention and transfer.

If the literature is clear on any point, it is that repeated testing under conditions in which retrieval is easy leads to poor long-term retention.

Not all retrievals are created equal: if I increased the interval modifier, I will review more frequently. More review leads to better performance (more successful recalls), but this doesn’t mean better long-term retention. In other words, if I increased the Interval Modifier, I might be wasting my time, but also hurting my long-term retention. So more work and worse long-term retention? What could possibly be worse than that?!

2. The Relationship Between Retrieval Strength and Storage Strength is Counter-intuitive

From “On the Symbiosis of Remembering, Forgetting, and Learning":

[Understanding the idea that] ‘the longer it took to answer a question, the more likely that answer would be recallable later’ requires understanding that retrieval is a learning event and the more difficult a (successful) retrieval, the more potent the learning.

In the new theory of disuse, the largest increments in both storage and retrieval strength occur when the to-be-learned (or relearned) information has low retrieval strength and high storage strength.

This is the concept of desirable difficulty: the more you struggle to recall the answer, given that you succeed, the better and stronger your long-term retention will be. So, increasing the Interval Modifier will hurt my long-term retention because they are making the reviews easier, undermining the effect of desirable difficulty.

Substantial improvements in performance across practice can occur without significant learning. - Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning

My point is, I don’t think I was equipped with enough knowledge to determine how to change any of the parameters, not to mention determining the optimal interval. We are all prone to cognitive biases: perceptual fluency, mistaking the sense of familiarity from performance for learning, misguided Judgement of Learning. Yet, I believed that the default setting is highly inefficient and at odds of what I learned. So I was trapped in the dilemma of knowing I had to change the default settings without knowing how.

What I Discovered From Tweaking the Anki Setting

Please see my post What I Discovered From Tweaking the Anki Setting for details about the default Anki algorithm’s behaviors.

On the other hand, in SuperMemo, the algorithm is mostly under the hood. You can only change the forgetting curve index. I have no control over how the algorithm behaves. To me, it’s a big relief. I don’t have to think whether I should change some parameters. No more wild guesses and then worrying about messing it up. I just have to trust (and I do) that the algorithm will take care of everything for me.

SuperMemo’s Superior Algorithm

Please see Is Algorithm in SuperMemo better than Algorithm in Anki? for details about my comparisons between different algorithms.

Introduction of Transfer of Knowledge

Please see Introduction of Transfer of Knowledge for a brief introduction of Transfer of Knowledge. In brief, achieving Transfer of Knowledge requires:

1. Increase your knowledge in long-term memory

2. Practice retrieving them

3. Somehow “load” “relevant” information into working memory

4. Somehow connect them together

5. Ding! There you have it: a new insight just popped up or successfully applied prior knowledge.

How do you use SRS, along with other learning techniques, to give you better probabilities of achieving transfer of learning?

Is summarization better than drawing a mindmap for comprehension?

Is self-testing better than self-explaining for transfer of learning?

And so, this began my quest for using Anki for transfer of knowledge.

My Frustrating Quest to Use Anki for Transfer of Knowledge

I’ve asked similar questions in r/Anki: How do I maintain the “big picture” of knowledge while using Anki? and Proposed solution: how to learn the big picture in concepts and knowledge with Anki

For example, I learned the concept of the Halo effect, read some examples from “Thinking, Fast and Slow”. Then in Anki, I’d memorize a few clozes of keywords about the Halo effect, summarize and come up with some examples. By doing so, I hoped to be able to identify new incidents and situations of the Halo effect in real-life, therefore, re-calibrate my judgements for better critical thinking.

I understand that what I was hoping to achieve was without objective measurement, as “big-picture thinking”, transfer of knowledge, creative insights are all vague terms without definitive criteria. Nonetheless, I quickly realized that just because I could recall those clozed keywords =/ I could recall that concept or idea when needed.

Derek Sivers mentioned such similar concerns in his interview with “Becoming a Superhuman”:

So my very first idea with these 220 book notes, and each one as you’ll see is pages and pages of notes on that book. My very first thought was like how do I put this into spaced repetition. Like how can I memorize what I’ve learned from these books with spaced repetition? And honestly, I haven’t figured out the answer yet. I actually played with some things. I did the- I think you call it the closure technique where you omit certain words. But even then, I was like, “Well, it’s not this sentence that I want to memorize. I’m not memorizing poetry. I want to memorize this concept, this idea so that it just becomes as internalized as one of those ones I said earlier, “whatever scares you, go do it” or your list of the 10 principles.

I think he is too, looking for achieving transfer of knowledge. The more and longer I memorized clozes of keywords for a concept, the more I felt like, well… I was just memorizing for the sake of memorization. These concepts seemed to exist in isolation. Knowing the existence of “The Outside View” or “The availability heuristics” was certainly helpful but I hoped to achieve more than knowing the names.

From A possible answer to the Q: Why is Anki great for remembering but not so for learning?, a member mentioned that:

However I switched to using SuperMemo and after a while I noticed a ton of connections being made.

Usually, I don’t trust anecdotal evidence because you could definitely find the exact opposite of such statement. Nonetheless, he understood my concern and what he was describing was very attractive to me:

I had a similar experience with capturing big-picture connections and SM helped address that.

Also, at that point, I was so frustrated with the Anki setting that I was willing to try something new. This reddit post was a nail in the coffin that made me give SuperMemo another try.

I’m well-aware that I was looking to achieve something probably outside the scope of Anki. No one could guarantee a better algorithm or Incremental Reading could help me transfer knowledge. But at least I believe it will help tremendously. Transfer of knowledge relies on more background knowledge. Since SRS plays a prominent role in long-term memory, a better algorithm for long-term retention should also help with transfer of knowledge.

Realizing the Benefits of Incremental Reading

Update: For more in-depth articles on IR, please see here and here

I have logs to record my SuperMemo use:

log 1 (March 22, 2018):

I don’t have to process the whole article at once. I really like the idea of processing it multiple times over a certain period of time. It’s distributed practice. When I go back to the article, I will have better understood due to the time lapse.

Log 2 (March 29, 2018):

There’s no more pressure to read and finish an article at once. I can just import articles into SM, and let it decides when it’s shown to me. I can also stop reading, press Next and revisit the same article later. Since the pressure to finish is gone, I can read a lot more articles. I can process the articles incrementally. INCREMENTAL READING FOR THE WIN!

When I was writing these logs, I was still early in my research about Incremental Reading, yet I could somehow see the potential of it; somehow the benefits and brilliance of Incremental Reading clicked. If you had asked me what was so special about Incremental Reading at that point, I probably couldn’t tell you. It was more of a sense that there was something very special about Incremental Reading.

In short, Incremental Reading is interleaving between reading and reviewing, and this greatly helps with comprehension and thus, provides a much better chance of transfer of knowledge. The magnificence of Incremental Reading is that it helps avoid quickly hitting the bottleneck of working memory. Working memory has a very limited capacity: try jogging between 5 pieces of new information and you’ll most likely lose your train of thoughts. The Magical Number 7+/- 2. From “Why Don’t Students Like School?":

If you try to juggle too many facts or to compare them in too many ways, you lose track of what you’re doing. Suppose I said, ‘What do a butterfly, a dragonfly, a chopstick, a pillbox, and a scarecrow have in common?’ These are simply too many items to compare simultaneously. As you’re thinking about how to relate a pillbox to a chopstick, you’ve already forgotten what the other items are.

On the other hand, Incremental Reading helps you chunk some of the information before further processing so you’ll have more room for thinking. For example, take the first few paragraphs of Chronic non-communicable disease risks presented by lipid oxidation products in fried foods. Supposedly, you’re new into this field and have no idea what PUFAs, MUFAs, LOPs, CFOs are. Continue reading could be very challenging and frustrating because as you go on, you’d frequently encounter those abbreviations and forget their meaning. During Incremental Reading, I would stop reading further and make the following items:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

Q: Full name of PUFAs?
A: polyunsaturated fatty acids

Q: Full name of MUFAs?
A: Monounsaturated fatty acids

Q: Which fatty acids are more susceptible to thermally-induced oxidation?
A: PUFAs

Q: Which fatty acids are resistant to peroxidation?
A: SFAs

It will probably be days or even weeks before SuperMemo presents me the article again. In the meantime, I will answer Q&As and clozes I made from the above paragraph. It’s also worth mentioning that spacing your reading and learning this way allows sleep to do its magic: consolidation of memory.

When I finally encounter the article again, at that point, I’ll have the basic facts understood and terms memorized. In this way, when I continue reading, I won’t be struggling with the basics. Having understood the basics means increased prior knowledge, and thus, decreased the workload of working memory, so that more of it could be allocated for deeper comprehension.

This is in line with #3 Build from the Basics of the 20 rules of knowledge formulation.

This is what Michael Nielsen does in Anki. He is doing Incremental Reading probably without realizing it. However, there’s one critical shortcoming regarding the spacing and interleaving between the article and its related items (cards) in Anki. Please see The Significance of Incremental Reading in SuperMemo: Part II

Closing Remarks

This article sums up my journey with SRS. Writing this is a great summary of how far I’ve come from reviewing with Anki to learning with SuperMemo. I’ve never regretted the decision to switch to SuperMemo ever since. I wholeheartedly believe that SuperMemo serves me better than Anki. I am very grateful for my wonderful 5 years with Anki and every experience in between.

Regardless of whether you are considering changing, I hope you gain something from reading. If I’ve convinced you to give SuperMemo a try, you might be interested in How I Moved My Anki Collection with Over 50,000 Cards to SuperMemo. Fore more please read SuperMemo: First Steps.