By Mark Gold, MD
Substance use has often been described as “bad learning” linked with impairments in reward processing and decision-making, but there is little substantial research to support this idea. A recent study by Byrne et al. suggests that substance misuse not only promotes harmful habit formation, which might undermine survival, but also makes it difficult to stop using.
Model-free vs. Model-based Learning
The “Dual Systems” theory of reinforcement learning defines two distinct systems:
The model-based, or goal-directed system, where actions are planned and purposeful, and we learn about the connection between actions and outcomes, and how to modify our behavior to achieve the desired outcome. This system requires more cognitive processing and is more flexible and controlled.
The model-free, or habit-based system, where learning is informed by reflexive responses to stimuli - like compulsive substance use and cravings. This system of learning is less flexible and is more controlled by automatic processing.
The differences between the two systems of learning have been highlighted by researchers in relation to harmful habitual behaviors such as substance use. One school of thought suggests that learning informed by the model-free system, with more of a focus on instinctual response to stimuli and less of a focus on conscious and informed decision-making, sets a person up to be more likely to engage in detrimental behaviors like substance use.
There is evidence that progressing from first use to misuse and addiction is paralleled by a shift from planned, purposeful, and goal-directed behavior to behavior that is habitual and reflexive. This progression and subsequent loss of control has been discussed by National Institutes on Drug Abuse Director Dr. Nora Volkow in her keynote speech at the APA and in her blog about free will. Model-free, conditioned learning means it is harder for a person to engage their frontal lobes, the part of the brain that helps us prioritize healthy, long-term and rational decisions. Repeated problematic substance use initiates a process where humans begin to respond more instinctually to the substance, wanting more and more of it over time. Use begets use, which leads to maladaptive behaviors centered around obtaining and using the substance to trigger the very same dopamine response that drives and reinforces model-free, habitual learning.
Substance Use and Reward Devaluation
Reward devaluation is a process that occurs in the brain where the value of a desirable outcome, like singing in a band, mentoring, or maintaining sobriety is reduced significantly. This process plays into why improving treatment outcomes can be so hard - treatment for addiction is not as “reinforcing” in the brain as substance use. Compulsive drug use is considered “highly pleasurable” by the parts of the brain that control decision-making when people are heavily addicted and feel as though they need the substance to survive. But treatment? not so much — long-term treatment is difficult to complete without continual support and a long-term treatment plan. Many patients stop attending treatment and/or support groups, and taking prescribed medications unless they are compelled to follow a set treatment plan and have adequate supports in place to help keep them on track.
Addiction is correlated to a considerable decrease in a person’s ability to devalue or disengage from habits learned through the model-free system. This means that problematic substance use affects our ability to make decisions and as the disorder progresses, we begin to put less value on long-term rewards and more value on immediately satisfying a need. Gradually, short term needs, like substance use, override long-term needs, like maintaining employment or investing in personal relationships.
Goals of Study
To examine the associations between model-based and model-free learning with a wide array of substance use behaviors. The process used to determine this was measuring individual variations in eye-blink rate, an indirect proxy for dopamine functioning, a key neural process related to model-free learning.
To assess whether problematic substance use predicted reward disengagement.
Why is This Important?
Patients with substance use disorders are driven to use despite harmful consequences, and although addiction is understood more and more as an acquired brain disease, many are still mystified as to why those suffering can't manage to break their “habit.” This study helps foster a greater understanding of the mechanisms that explain why. Use may be thought of as “recreational” by the user, but it poses a challenge to the brain, reinforcement systems, and reward hierarchies, which can change a person quickly and in a way that is hard for those around them to understand. Once reward-outcome associations are well established— i.e., taking drugs makes a person “feel good”— individuals with substance use disorders have changed the most basic mechanisms in their brain, and will have more difficulty disengaging from the habitual tendencies. It is not clear how individual experiences, genetics, trauma, and other factors change the speed of these changes. That said, the results of this study are consistent with previous data depicting how alcohol dependence indicates a greater likelihood that a person has habit-based learning strategies over goal-directed strategies. The results do not, however, provide us with more information about whether biological recovery is possible, and how we could make recovery more likely and sustainable for patients.
Authors state that current findings highlight how problems with substance use go beyond the realms of habit formation: they also influence the process of disengaging or “breaking” habits by making it more difficult for individuals with substance use disorders to stop using substances. A better understanding of the mechanisms in the brain that take over once substance use becomes problematic may help us create more effective prevention campaigns and treatments once substance use progresses to a harmful habit.
Dr. Mark S. Gold is a teacher of the year, translational researcher, author, mentor and inventor best known for his work on the brain systems underlying the effects of opiate drugs, cocaine and food. Read more by Dr. Gold here.