Misgivings about SRS
Earlier this year the Global Times did an article on using SRS (spaced repetition software) technology to “Learn Chinese in a flash.” The journalist interviewed both me and Dr. Orlando Kelm about the issue, but most of what we said didn’t actually make it into the article. I’m going to use the content of that exchange to finally address my misgivings about SRS.
My SRS misgivings are grouped into three main points below, and I’ve added in some of Dr. Kelm’s input, with his permission.
SRS is a way to enhance your language studies, not a substitution for them
Back in the good old days, we students used to take our vocabulary lists and make flashcards out of them. As we amassed stacks and stacks of these flashcards, it was hard to systematically review them properly, and to keep track of which stacks of cards had which vocabulary. SRS completely solves this problem with a tidy little review algorithm and a feedback mechanism which you interact with as you review your vocabulary. This is great. Those of us who were too lazy to create stacks and stacks of flashcards can now feel vindicated; we will never have to, because technology has saved us from all that arduous flashcard management.
The problem, however, is that SRS is sometimes over-emphasized to the point that it almost seems like a “language acquisition method.” Especially for the analytical-minded, it can be easy to get lost in the efficiency of the review system and all the pretty stats, forgetting that memorization of vocabulary is only one part of language acquisition. If the SRS-obsessed student is not getting plenty of natural target language input and speaking practice, he’ll end up the linguistic equivalent of the guy at the gym with bulging upper body musculature but pencil legs.
Dr. Kelm warns against the “one method for everything” approach as well:
It seems like every time we discover something that is good in one area (e.g., SRS that helps in aiding rote memorization) the tendency is to try to apply it in every other area (e.g., speaking a foreign language). I have seen the same thing with lots of second language theories. For example TPR (total physical response) is a theory where people are supposed to physically use their senses while learning a language (actually open a door when saying “I open the door”, actually taste the food when they say “I am eating a banana). Great, TPR may be OK in some instances, but then people try to apply TPR to every aspect of language learning. It just gets crazy after a while. To me the same issue comes up with SRS. Just because it is good for rote memorization, doesn’t mean that it will be good for all aspects of language learning.
Dr. Kelm reminds us about what else is important that is outside the realm of SRS :
The biggest issue here, as related some of the limitations of SRS, is that of input vs intake, schema theory, and scripts. A gigantic part of language learning is related to CONTEXT. I’m sure there are times when you can recall the exact moment when you heard a new phrase in Chinese, learned a new word, or did something in another language.
For example, last month when I was in China a seller came up to our car and asked my guide if he wanted to buy something. All he said to the seller was “mai bu qi” (I can’t afford that). For me it was the perfect moment because I saw how a native speaker reacts to the sellers. Where I would have just said “bu yao” or “bu yong“, there was something cool about hearing “mai bu qi“. The phrase stuck in my mind and I’ll be able to use it from here on out. This is a great example of how context affects our learning. The more we can create context for learners, the better we retain the foreign language. Note that this is not related to frequency of occurrence or frequency of review (principles of SRS), but more to the impact of the moment. SRS doesn’t necessarily take this feature into account.
Second, language learning also happens in chunks and people learn these chunks in scripts that we follow. For example, if you go to a fast food restaurant to make an order, at some point the cashier will say “Is that for here or to go?” You know the pattern, you expect this question to come up, and so you are prepared to answer it. Even if you don’t hear the question exactly, you can still guess at what was said. It’s part of the “script” that we all follow when ordering fast food. When language learning relates to these chunks and scripts, it helps to make things stick. Note again that this is not related to the frequency of occurrence and isn’t where SRS will shine. (I should probably add that ChinesePod does a really good job of creating short dialogs that help provide this context and simulate these scripts. They recycle vocabulary in various contexts well.)
SRS and the DIY factor
Creating flashcards is a meaningful activity in itself. The act of creating the cards, with each word carefully scrawled by the student (and maybe even a picture or two!) contributes to the learning. Anyone who has ever used flashcards can tell you there’s a big difference between making your own and buying pre-made flashcards.
Ideally, the words and sentences added to your SRS come from your own experience, or from the material you are personally interested in studying. This makes the learning more personal and the results more satisfying. Many students, however, are reviewing ready-made vocabulary lists, pre-loaded into the SRS. This type of review isn’t worthless, but because the learner’s degree of involvement is so much lower, each word’s “memory imprint” is much fainter. It’s also much easier to simply toss aside and forget a digital “stack” of flashcards that took 3 seconds to download, compared to a personalized list one has invested time and effort into.
Using SRS well is a skill
This is the part that no one really expects, because it’s nice to think that technology has solved our problems. The truth is that using SRS effectively is an entirely new skill. I mentioned already that ready-made decks are less likely to be effective, but even an active learner carefully looking up new words and adding them to SRS (with some context) can easily go wrong.
I’ll give you a personal example. I was reading a Lu Xun story, and it contained a fair amount of vocabulary with which I was unfamiliar. After looking up the new words, I dutifully copied them into Anki (my SRS client of choice). There was a fair amount of vocabulary just from that Lu Xun story. Over time, I found that the Lu Xun vocabulary just wasn’t sticking. The words were semi-archaic, and I had virtually no chance of running into them in my modern daily life in Shanghai. I found they were useful only for reading Lu Xun (or possibly other Chinese literature of that era), and yet I wasn’t spending a lot of time reading that literature. The vocabulary was effectively “clogging up” my SRS review sessions as I had to repeatedly review those words, which meant I had less time to spend on review of more useful vocabulary, and I was rapidly losing motivation to use SRS altogether. When I found myself going a week or more without doing any review at all, I eventually realized that I had effectively killed my review sessions and needed an “Anki Reset.”
Including too much obscure “recognition only” material is not the only pitfall; other typical mistakes include lack of sufficient context, overly long sentence examples, and insufficient consideration of what is actually useful in one’s active vocabulary. It’s the memorization of vocabulary which one is able to actually use in conversation that is the most satisfying, after all. Failure to accomplish this essentially amounts to “vocabulary hoarding,” not proficiency in the target language.
Since using SRS properly is a skill which must be practiced, it demands time in itself. Learning to use SRS well and getting into the the habit of using it will take time, which could otherwise be devoted to listening or speaking practice. Is it worth it? For some, the answer is an unabashed yes, yes, a thousands times YES! but for many students the answer is not so clear-cut.
Great post, and it addresses the problems I’ve had with SRS. I’m lucky enough to have the sort of rat-trap memory that makes learning new items a bit easier, but it’s almost entirely dependent on context; this may be why I’ve never had much use for flashcards. Context is also key, at least for me, in remembering the tone of a given word or character: I’ll refer to sentences in which I’ve heard the word used, rather than to the character’s dictionary entry.
I’ve also got some misgivings about the usefulness of digital ‘learning aids’ in general. Tools like Wenlin and nCiku are great as references, but I’ve found that too much convenience can actually be a bad thing when it comes to retention of new vocabulary: the added inconvenience involved in looking up a new character or word in a print dictionary actually helped to fix the word in my mind — this may go back to context — whereas an instant definition is less likely to stick in my memory. The same is true of digital flashcards: the rote repetition of writing something out by hand multiple times will almost always ensure that I remember the character; typing the character in using a Pinyin IME, not so much. It’s not that these tools are bad things, and there are certainly plenty of language learners for whom being able to hand-write characters is not a priority, but in my own experience, there’s something to be said for keeping it old-school.
Also: I’m totally with you on SRS use being a skill unto itself — this may well be the real reason I’ve never made any use of it. It strikes me as being sort of like Getting Things Done: those who have bothered to teach themselves the new system swear by it; the rest of us just swear.
Yeah, I know what you mean. I totally agree. I can remember carrying my paper Oxford dictionary around with me, and then hearing new words, then thumbing through the pages, searching for what I thought I heard. It was a process of discovery, of trial and error, and above all, of learning.
Still, a part of me can’t help but wonder if this reaction isn’t just the old man in us saying, “when I was a youngster, we used to walk 5 miles to school, uphill, in the snow, both ways.” As technology makes scarce certain learning methods, it simultaneously adds new ones.
As usual, there is a balance to find. I tried the reader of Pleco and found that I couldn’t read anything on it, because it was so convenient that I would check almost every character and word, even the ones I know. However, I hope I will never come back to using a paper dictionary (I did do that for two years while in France). It is a waste of time, and I don’t think it really serves any purpose once you know the radicals. What I do is to buy a book or to print pages and then read them using the character recognition of Pleco to find new words. For me it is the perfect balance between usability and laziness.
As for SRS, as rm says, it is better for remembering words than for learning completely new words. I put a huge amount of pre-made cards on it (the HSK vocab lists) because I already know most of these words, but my knowledge is not complete (I would forget the tones or how to write a character). In my opinion, SRS is good to solidify this kind of vocabulary. And I completely agree with John that SRS should be complemented with a lot of input, and preferably output.
Good post, John.
Mostly I’d echo what Brendan said already. But since in contrast to the O’Kane I have a mind like a wide-bore sieve, I have used SRS with some success, specifically for recognizing single characters without any in-word or in-phrase context.
And to add my agreement that “SRS is a skill”… I had to tinker with it a lot before I figured out what to use it for and what not. And about two months ago I abandoned it altogether because I started thinking that the hour a day I spent could be better used at doing something else that promotes lots of repetition: i.e plain old reading 😀
I rely a huge amount on SRS, but I think you’re right to advise caution. I used it to learn how to write characters — for me, this was more a rote-memorisation task and SRS worked great.
But for regularly learning decent-sized amounts of vocab, I try to make sure I use SRS for remembering rather than learning. That is, I’ll try to learn the word before putting it into the SRS. If I’m not happy or comfortable with the word before putting it in, then I won’t put it in.
I think that makes sense given the model these systems seem to use: that the initial gap between seeing a card the first and second times is around 4 days. So you’re expected to have learned it at least reasonably well before putting it in.
Lu Xun cards do not belong in an SRS. I mistakenly put in cards from the story 祝福 and never came across most of those words anywhere else. You know you’re in trouble when the example sentence from the dictionary is the same source as you used!
After spending several years working with heavy paper dictionaries, I have quickly fallen in love with tools like Pleco and Wenlin. Particularly useful is the ability to write in the character you’re looking for and skip the bother of radical lists that may or may not have your target item. Still, I agree with Brendan that the effort of looking words up did help in retention, as well as understanding the structure of characters. Perhaps an even bigger loss for me is not having pages with upper-lower, left-right quadrants. I have found that having a visual image of the location of words on a page is very helpful in retaining them. Hand-written flashcards can occasionally have the same benefit by providing spacial or aesthetic relations–I sometimes would remember a word due to the fact that I wrote it especially poorly or it was scrunched up against the edge of the card. Electronic dictionaries and flashcards with endless scrolls of words and perfectly formatted word boxes do not offer this memory aid.
[…] this, learning words without context is not a perfect option by any means. Sinosplice has a good post about the limitations of SRS. My method is to try to use a search engine to give the words I am […]
[…] three great articles about SRS. In the first one, Misgivings about SRS, John Pasden explains as some people, especially the ones that have a very analytical mind, fall on […]
[…] tre articoli interessanti sull’SRS (in inglese). Nel primo, Misgivings about SRS, John Pasden spiega come molti studenti, specialmente quelli che hanno una mente analitica, tendono […]
I have become a hopelessly-addicted SRS user in recent months. This decision came at something of an impasse in my (nine year-long) Chinese language-learning journey, and was made largely on the back of blog I came across, the author of which was positively evangelical about the possibilities of the technology.
By now – nine months in – I recognise all of the problems and limitations cited above. I was mistaken to think, as many others have, that SRS was a cure to all language-acquisition ills. It is bound to unnaturally skew one’s priorities and lead to the kind of imbalanced result you allude to in your post (ie. I have bulging vocabularly pecs, and puny grammatical legs). That said, it has proved useful in certain respects, not least in introducing a competitive element to language-learning (albeit one in which I compete with myself) and imposing quite a hard-edged discipline (ie. I gotta get through my character sets every day, regardless of how I feel, otherwise the ‘overdue cards’ count mounts very quickly….this can verge on the pathological).
My current set up attempts to address some of the deficiencies mentioned above. Though it’s probably very, very boring, I’ll set out my current arrangements, as briefly as possible, in the hope of explaining how they work for me (and occasionally, how they do not).
*I have four decks of cards which, in total, I spend around an hour trawling through daily.
*Only one of these, the HSK deck, was downloaded and, as such, contains many words and (at Level 6) idioms which are completely devoid of context for me. Because of the sheer size of the contemporary Level 6 HSK category (1,400+ words), I have had to introduce new cards slowly – I try for 10-20 new words per day – in the hope that by the end of this year, 2012, I will be juggling all cards (about 2,500 for Levels 1-6), while never having to face a single daily session of more than, say, 150 words at one time.
*I download daily audio from YouTube clips of 美国之音 TV news broadcasts and listen to them as MP3 files whilst commuting, or taking a stroll. I attempt to listen to at least 20-minutes worth of broadcast material daily. Additionally, I force myself to read at least one Chinese news article (I occasionally substitute this with a page or two of a novel) per day, regardless of subject. These two activities have allowed me to locate the usage of a lot of the fairly formal words or obscures idioms that I have come across in my HSK drills (especially when I read Chinese newspapers, as these are the most likely to feature the more obscure, Mao-era, political terms often used in the HSK). I don’t always have time to dwell on their exact usage – and there are many words/phrases I have not yet heard in any real-world context – but I do get a little thrill when I hear a word or phrase which I have previously only known in the HSK context, being used out there in the real world.
In short, I try to undertake the (largely written) daily SRS drills in tandem with attempts to exercise my listening and reading skills.
*My second and third flashcard decks are drawn manually from Chinesepod.com. I listen to lessons at the Intermediate and Upper Intermediate levels (keep up the good work, btw!:)) and, after each lesson, draw down new words/phrases into files which I transfer to my SRS system (Pleco, for what it’s worth). Thus I have an ‘Intermediate’ set, and an ‘Upper Intermediate’ set which are both increasing in size on a weekly basis, as new lessons are made.
*My fourth flashcard set – and the most recent, and possible useful, addition – is a list of complete sentences which locates some of the most common/useful/interesting words/phrases in real-world context. I tend to take these sentences from the dialogues at Chinesepod.com, thereby ensuring that they are reliable in terms of how people really speak. This is an attempt to address the most obvious failing with SRS that it allows you to expand your vocabulary without requiring any understanding of how words are actually used in context. In this test, I look at the English translation and read out the correct Chinese sentence. The act of verbalising, if only to myself, seems to make certain patterns stick.
*In terms of the specific tests that I undertake, I oscillate fairly systematically between, on the one hand, viewing the English translation and responding with the written Chinese translation (input using hanzi), while simultaneously verbalising the word in the (hopefully) correct tones; and, on the other, reading the Chinese word and verbalising the correct English translation out loud to myself. Regardless of the exact test I undertake, I try to be disciplined and have a rule for myself that if I could not, on request, write the hanzi that appear in the word, or if I get the tone of a character wrong (even if I knew how to write it), I mark the card as wrong. In some ways this is a vanity project – I want to be able to say (as I have been known to in the past) that “I am able to write everything that I am able to say”. On the other hand, as some other commenters have noted, writing a character over and over again does tend to make it stick in one’s memory banks.
As I mentioned, all of this takes me between 60 minutes and 75 minutes per day.
Despite all of my labours, I have concluded that while daily SRS work has enlarged my vocabulary and improved my reading skills (and to a lesser extent, listening skills), it has done absolutely nothing for my general conversational fluency. If anything, this is in a worse place now than it was nine months ago. I lived in China for several years in the Noughties (apologies:)) and, thus, feel confident in terms of my basic pronunciation and tones. But, here I am, nine years in, still finding myself jumping through all kinds of mental hoops and using torturous (and probably way overly complication and clunky) sentence constructions when it comes time to actually have a conversation at anything over a basic elementary level. Similarly, I have little confidence in composing a Chinese sentence in writing. I may be able to write the individual characters accurately, with the correct stroke order etc.etc, but I cannot necessarily link them fluently in a proper sentence, let alone a paragraph.
In summary, SRS is rubbish for improving fluency, but is great for developing vocabulary and thus (depending on precisely how it is used), improving one’s reading and listening comprehension. Luckily for me, right now I am most concerned with improving my Chinese reading skills, so this works for me. And I am (semi)confident that this is great foundational work for when I do, eventually, get back to China and find myself speaking with real people again (you currently find me residing in a sleepy English village – which, over and beyond everything I have said, is my biggest problem of all – the general ambient sounds in my everyday life are not those of Mandarin Chinese!)
Anyway, that’s probably quite enough. Sorry for length of this post. And thanks for your thoughts on SRS, John (albeit two years after you made them). Very interesting stuff.
I’ve been using SRS for quite some time now, and I agree with the general assessment: it is a powerful tool which must be used wisely and for specific purposes. In this case, SRS is most excellent for building passive vocabulary – this is my focus. But one thing that everyone seems to forget is that learning a foreign language is the most complex skill to learn and the most taxing on memory.
Only recently I’ve begun to incorporate memory techniques with SRS. I encode the memory of a data set before I enter it into an SRS file. Encode here means to over learn a data set by using memory techniques such as Mnemonics, Association, Memory Palace and Pegging – the four classic techniques. Sometimes I use Mind Maps for larger data sets such as sentences. Memory experts all agree the better you encode a data set, the better chance of both recognition and recall.
From my experience, recall is very difficult for language learning. However, the memory techniques have aided my recognition of an item after I’ve added the data set to my SRS files. Because I have over learnt the data set, the “spacing” is very effective.
What follows is a technique which I am currently using for vocab. These are not my words but I am quoting. If used in conjunction with SRS it is a very effective technique for RECOGNITION – or better put, passive vocabulary.
Recall 5-1-3-6-3 for 2
Step 1. Take in the information or content and apply memory techniques such as Association
Step 2. About five minutes later, go over the content of what you are trying to remember, taking a minute.
Step 3. An hour later, do the same thing.
Step 4. Three hours later, do the same thing. Just go back over the information a minute or two.
Step 5. Six hours later, do the same. That night before you go to sleep, review the material one last time.
Step 6. Repeat three times a day for the second and third days and you have that information for the long term.
5 (min)
1 (hour)
3 (hours)
6 (hours)
3 (times) for 2 (days)
After you have done this, if you are still struggling with the data set, then I recommend you add it to your SRS. By this stage, you have have over learnt the data set and encoded it deeply. In fact, you have spent a considerable amount of time and attention on the data set. The spacing should be very effective.
All memory experts agree that the first stages impressions of a remembering are the most important. You need to give a data set attention and time. They also agree to over learn a data set. Otherwise you are in for the long haul of constant reviewing.
If anybody adopts this method, then please share your thoughts.
[…] commenter Graham Bond recently left a lengthy and interesting comment on my Misgivings about SRS post. (“SRS” refers to spaced repetition system like Anki; I […]
[…] primero, Misgivings about SRS, John Pasden explica como algunas personas, especialmente los que tienen una mente muy analítica, […]
SRS can definitely turn vocabulary into abstract knowledge if you aren’t actively using it in context and connecting it to other things you know and understand. For this reason, I really appreciate the way that SRS is incorporated by Memrise. The creation of mnemonic devices a) helps to personalize the word for you, even better than physical flash cards, and b) helps to attach the new abstract information to what you already know and use through the mnemonic device. This is best when you take the time to create the mnemonic yourself, but it is also valid if you use one that someone else created on the Memrise system. So the mnemonic helps to embed the new concept in your memory and the SRS manages the long-term retention. It’s a very useful approach. (I don’t work for them, just really appreciate what the software is doing for my vocabulary acquisition.)
[…] so doing occasional reviews pretty much solves that problem, right? Well, maybe… SRS drawbacks aside, certain personality types like to take a more active role in the vocabulary categorization […]
Some people commented that using an SRS can create an imbalance between the size of your vocabulary on the one hand, and grammar (and fluency etc.) on the other. But why should you use an SRS only for vocabulary? It is perfectly possible to create a flash card for John’s 买不起 example. It is also possible to turn sentences into cloze tests for certain grammar patterns. And Anki also allows flash cards with sound recordings on the front (you can get them at Forvo.com and Rhinospike) and responses or transcripts (depending on what you want to test) on the back. And for vocabulary flash cards, you can always consult a frequency dictionary before deciding whether you want to add a new item. (Of course, buying yet another book may worsen the “bookshelf problem” that John described in another post.) Gabriel Wyner’s book “Fluent Forever” that an SRS can be used for more than just vocabulary.
For conversational fluency, you will need to get away from your SRS, as many people pointed out. If you don’t have easy access to native speakers, you can use sites such as Verbling, LiveMocha and iTalki. Don’t blame your SRS if you don’t make enough progress in this area.
[…] lot of potential. Basically, “Learn Mode” is FluentU’s take on SRS, an idea which isn’t so great all by itself, but holds a lot of promise for enhancing other methods of […]
Chinese language is not that easy to learn. My sister went to China for 5 years but still not fluent in speaking chinese.. .
[…] three great articles about SRS. In the first one, Misgivings about SRS, John Pasden explains as some people, especially the ones that have a very analytical mind, fall on […]
[…] As a beginner, you may have a hard time knowing what to add now and what to leave alone for now. To get really good results you have to choose the right words to add, and knowing what to add is a skill. […]