Enhancing Students' Communication Skills Through Machine Learning
Why should students focus on feedback?
I today’s day and age of online learning, which spans across continents and cultures, one’s ability to effectively communicate with confidence comes to the forefront. In fact, 90% of parents chose communication as the highest priority skill for their children to have (Pew Research Center). However, like many other skills, communicating with confidence does not happen overnight. It takes iterative improvements & practice based on quality feedback from experts. In order for LearningLeaders to achieve our mission of inspiring 1M+ leaders, we must enable learners by providing actionable, skills focused feedback.
Why is LearningLeaders best positioned to help students communicate with confidence?
LearningLeaders is best positioned to create the leaders of tomorrow for multiple reasons. Firstly, our experience over the last 8 years of coaching speech & debate not only includes helping students win World Championships, but more importantly, the earned insights as to how to get there. Secondly, our skills first feedback system is the first of its kind in this space, and ensures that our students’ success is lifelong and outcome oriented, not just a flash in the pan. Thirdly, our partnership with machine learning experts Obviously AI allows us to make data-driven decisions & recommendations, while finding patterns in our students’ behaviours beyond what is immediately obvious. Additionally, we pride ourselves in taking no shortcuts, especially with our students’ learning experience. This means not offering off the shelf feedback and not delivering “one size fits all” styles of coaching. There is no trade-off when it comes to our students. These combined factors make up our analytics strategy, which is underlined by the following premises:
Effective communication (defined as both clear and persuasive) can be broken down into component behaviours (ie. eye contact during interaction is more predictive of effective communication than lack of eye contact). What defines effective communication may change depending on the scenario in which it is desired.
Behaviours are performative demonstrations that are lagging indicators of skill acquisition (ie. when a student is able to outwardly display certain behaviours repeatedly, that student is said to have acquired a skill). The higher level of mastery of a skill achieved, the more likely the person is able to display that behaviour effectively at the right moment.
In order to effectively coach and hone skills over time, certain methods are higher-value (more effective and efficient) than others. These methods may change depending on the individual, their age, culture, etc.
There have been many studies within education that apply machine learning techniques to student data for predictive or analytical use. However to our knowledge, none within the public speaking & debate space. In this paper, we attempt to model outcome oriented communication skills, and competitive success. Like Billy Beane did with the Oakland Athletics when using analytics and evidence based decisions in player recruitment, we will attempt to do the same within speech & debate (hence the paper title). Our hypothesis is that there is a relationship between students demonstrating unique specific skills and their probability of competitive success. The null hypothesis is that there is no relationship.
Why Now?
You’d be hard pressed to find an industry that does not leverage data to delight & inspire its customer base. The migration from offline to online learning over the last 2 years is unlike anything we’ve experienced before. With the introduction of new technologies such as video conferencing, Artificial Intelligence, and Virtual Reality, the power of enhancing the live in-class experience has shifted towards tech & data. It’s clear that the COVID-19 global pandemic was the catalyst in this movement but it shows no sign of slowing down.
By 2025 Augmented Reality (AR), Virtual Reality (VR), and Artificial Intelligence (AI) will have become increasingly integrated into core education delivery and learning processes(i). Advanced technology expenditure in global education in AR/VR will hit $12.6B by 2025, compared to $1.8B in 2018. Similarly for Artificial Intelligence, expenditure will reach $6.1B by 2025, compared to $0.8B in 2018. Although LearningLeaders has always been data driven, our current research, data, and technology now allows us to dive deeper, and truly “Focus on Feedback”.
Unfortunately, however useful AI tools in education may be, they’re not always beneficial to students. For example AI-based writing tools such as GPT-3 & Grammarly can make it very hard to decipher how much a student contributed, and how much of the contribution was the machine. Therefore, we understand the true impact of technology on students is when our AI generated insights are applied to live communication with students in the classroom or competitions. In this way the technology is used to guide and support towards lifelong learning, not to offer shortcuts.
What issues are students facing?
Feedback has been and always will be fundamental to students’ progress in learning any skill. Realistic, specific, and high quality feedback that identifies both strengths & weaknesses, coupled with students' own active reflection (self-assessment) can transform their learning experience. What’s also true is that effective communication comes in many forms, and differs from student to student and situation to situation. Therefore, we are formulating a new and improved way for students to take their learning into their own hands by using LearningLeaders’ predictive insights to improve on their past performances. The end goal is to become outcome oriented communicators. This is to say that our analytics strategy uncovers clearly defined skills and metrics that students can then prioritise and master in order to achieve effective communication, be it in competitions, school, their future career, or everyday life. What we didn’t know when we started but now know is that 90% of parents believe that communication skills are essential for their children’s future. Also, communication skills are also the single most frequently-requested skill from hiring managers around the world on job boards. So why aren’t communication skills & effective feedback easily available for students? If they are, how do we ensure students benefit as much they can from speech and debate education?
How can we help?
Speech and debate provides a unique case for testing communication skills with steady-state outcomes. The LearningLeaders Research Institute will leverage the existing skills-based progression system to generate predictive analytics of student performance, skill acquisition, and ultimately help LL design improved learning methods and platforms over time.
To win debating rounds in a competitive environment, a team needs to perform comparatively better than other teams. Each round is observed by an independent adjudicator that evaluates teams and individual performances. But how do judges determine that one team has performed better than another? How do they know which team has won or who is the best speaker in a particular round? Regardless of the format, judging manuals outline different types of metrics that judges should use when evaluating rounds. For instance, at the World Universities Debating Championship, judges are required to use the following criteria when evaluating a speaker’s performance: quality of analysis, style, general knowledge, quality of engagement with other teams, comparative analysis, capacity to meet one’s own burden of proof and others. To have sound performance and achieve the best result under a specific criterion during a round, speakers need to present mastery of a wide set of skills in a way that is better than their competition. For example, let us take the criterion analysis. In its broadest sense, the term describes all of the reasons that a speaker offers to support their stance or claim in the round. According to the WUDC judging rules, to be successful in their analysis, speakers can: use sound logic to explain why their arguments are true, use empirical evidence to support their claim, expose a damaging logical implication of a contrasting argument, or use various other techniques that encourage the judge to believe that an argument is true and important to the debate. The rules also state that speaker’s reasons may be stronger or weaker depending on the detail of explanation, precision of expression, moral intuitions and logical implications. Based on these criteria, judges will try to evaluate analytical contributions holistically and decide which team has been more successful at fulfilling their role in the round. What does this mean for debaters and educational designers of speech and debate curricula?
If judging manuals, among other things, set expectations that one needs to provide sound analysis in their speech, speakers must therefore show the ability to perform this particular skill in the round. Thus, demonstrating mastery of communication skills eventually leads to a ‘win’ or a high speaker score, which is each speaker’s indicator that they have effectively performed in the round.
To get debaters ready to perform, curricula designers and coaches need to identify the most important skills that would help students win rounds and build lesson content that would help them acquire these skills. Curricula designers would also need to understand which skills are more important than others and distribute their efforts in such a way that would help students focus more on key skills and less so on non-crucial ones. For example, say one of the first criteria judges look for in a round is whether a team has fulfilled their burden of proof. Then they may focus on logical analysis that leads to a team fulfilling their burden. This probably means that teaching efforts should focus more on how to understand and use burdens of proof and reasoning in a round , and less so on non-crucial skills for competitive success such as eye contact. With that being said, how can speakers and debate coaches know that a speaker is improving their skills? Assuming a speaker acquires the ability to support their claims with logical reasoning, how do we track their progress and understand their level of mastery of this particular skill?
Regardless of tournament results, one could make an argument that every debater acquires a valuable set of skills through debate rounds and regular practice. To have higher success rates in debate competitions, students must perform comparatively better than their peers. This means that they need to be more skilled at reasoning, note-taking, processing of information, meta-analysis, and other skills that judges find important. By identifying a key skill set and understanding how to best track student progress, curricula designers and debate instructors can provide more nuanced and accurate personal feedback for speakers, while the lesson design matches the individual needs of each speaker. By creating evaluation metrics that we can successfully use to keep track of student performance, we are better able to keep track of student progress, estimate efficiency of teaching methodology and better predict learning outcomes. Consequently, a more nuanced teaching approach that revolves around improvement of skills would lead to skill acquisition and improvement. This would help students achieve greater competitive success and education outcomes.
In total we have collected data on 1138 speeches across 5 different debate competitions.
These competitions are organised by LearningLeaders which ensures consistency during the data collection. These competitions include the World Schools Debating League, International Competition for Young Debaters, Cambridge Asia British Parliamentary, and Shanghai Public Speaking & Debate Championship in both WSDC & BP formats. Through an iterative process, our experienced debate coaches boiled down the BP & WS formats into 10 specific skills that they deemed were essential and fundamental to debate. These skills are clearly defined within our curricula and are as follows.
- Contextualizing and Framing
- Rebutting Comprehensively
- Using Rhetoric to Build Emotional Appeal
- Building Plausible Impacts
- Building Relevant Arguments
- Building Strong Reasoning
- Using Appropriate Pacing
- Weighing Impacts
- Contributing Actively Through POI’s
- Delivering a Structured Speech
The model used for this analysis is a Voting Classifier model which is a collection of different classifiers that averages over the prediction results for each as the final predicted category. This machine learning algorithm is built to predict a competitive outcome of Win or Lose and has an accuracy of 80.18%. This metric represents the accuracy of the model on the Test set. The Test set is the portion of the data which the model has never seen before and is used to evaluate the model's performance. The accuracy value means that when you test this model by uploading data of 100 random or new students, you can expect the model's predictions to be correct around 80 times and incorrect around 20 times. Let’s compare predicting a round winner to a coin toss. Given a round between two random unknown debate teams, the probability of predicting a winner would be 50/50, similar to that of a coin toss. However, using this new-found information about skill deployment in debates, given the team data, the probability of predicting the round winner becomes 80/20. We therefore see an increase of 60% in the probability of predicting the winner.
We can break the results into a variable’s contribution to the model, and its impact on predicted outcome. A variable’s “contribution” to the model can be seen as its measure of how much the model relies on it in order to make predictions. The higher the contribution, the higher the influence it has on predictions. For example, in a model that predicts tomorrow’s weather, we’d most likely rely on today’s weather more than what day of the week it is. However, this does not tell us the impact today’s weather has on tomorrow’s weather, only that we should rely on it more compared to other variables. Today’s weather could in fact have a high contribution to the model but a negative impact on the outcome i.e. hot humid weather today, thunderstorms tomorrow. Therefore, It’s important to note that contribution is a different metric to “increases probability of winning by x%”. Contribution can be seen as how important the variable is to the model (better predictions), whereas its effect on the outcome can be seen as how important the variable is to the student (better outcomes).
- Rebutting Comprehensively contributes 5.4% to outcome and having this skill increases your chances of a win by 26%. In fact, not having this skill will actually increase your probability of losing by 5%.
- Weighing impacts contributes 2.2% to outcome and having this skill increases your chances of a win by 26%.
- Contextualizing and framing contribute 2.9% to outcome and having this skill increases your chances of a win by ~22%.
- Building Relevant Arguments contributes 3% to outcome and having this skill increases your chances of a win by 15%.
Practically speaking, based on this preliminary analysis, in order to maximise probability of winning a round, students should first ensure they are exhibiting strong use of rebuttals above exhibiting other skills. In order to do so they should drill this with their partner(s), focus on coach & judge feedback, and actively ask coaches for support in this area. This focused work may result in an additional 1 win for every 4 debates!
There are other driving factors in the prediction of competitive outcomes for students, such as speaker role, debate round, which competition format, and many more. However, for the purpose of this research we will be focusing on the skills and the driving effect they have behind competitive success.
Thanks to our machine learning partners Obviously AI we now have the ability to offer instant & live predictions on our Learning Platform by integrating this model with their classroom performance data. This will offer students a benchmark as to their learning level as they enter our program. They will then be able to use our feedback system to track their progress. This framework of feedback can also impact students in the following way:
- Help students better understand their goals by offering extremely clear metrics, as apposed to a judge manual or speaker score.
- Feedback is far more targeted which complements & condenses the longer form feedback offered on ballots & after rounds as to not overwhelm students.
- Immediacy of the feedback as mentioned above. Students can see how they have improved their chances of competitive success after each round in lessons.
Debate aims to achieve several different goals. It aims to create advocacy-oriented education, teach persuasion skills, teach critical thinking, create positive classroom atmosphere, create new patterns of knowledge, and create a fun atmosphere. As these goals are achieved, students receive a wide palette of skills that they can use in a variety of different segments of their lives.
Through debate, students develop their competitive mindset, as well as technical skills, confidence and teamwork skills, all attributives highly valued by employers. The existing research in educational psychology gives us a strong reason to expect that these benefits will only increase as debate pedagogy is implemented across curriculum. This in turn will help a huge number of students develop their reasoning skills, which are essential to understanding the world. Increased knowledge of argument building, results in increased knowledge of the reasoning process and this should help reinforce learning. As the world’s population becomes better educated and literacy levels rise, the demand for improved argumentative processes increases.
One of the main skills that a student needs to master in order to become a successful debater is critical thinking. In essence, critical thinking means analysing and synthesising ideas, going below the surface of an argument, articulating unstated assumptions and testing validity of ideas. How does critical thinking impact education outcomes for students of speech and debate? Critical thinkers learn how ideas relate to each other and understand the importance of logical consistency. Critical thinking involves analysing problems, selecting and examining evidence, interpreting data, determining logical relationships, testing reasoning, reaching conclusion, and selecting appropriate language. These are all critical skills for making good rational choices when confronted with a large amount of information. The most frequently cited perks of participating in competitive speech and debate activities, generally shared in this order, include improvement of critical thinking, communication competency, college and employment prospects, and teamwork and relational skills. Listening to debates generates benefits as well: spectators often learn a new way of thinking about a problem or an issue. These observations might feel intuitive and clear, but what is less obvious is the extent to which the acquisition of speech and debate skills has a positive impact on educational outcomes for students.
To understand the answer to this puzzle, we should look at available research results on the impact of speech and debate on education outcomes. In a study conducted in the Houston Independent School District that covered over 35,788 students (amongst whom there were 1145 debaters and 34643 non-debaters) researchers tried to identify a causal link between debate participation and student achievement. According to this research, debate participation was associated with higher GPA scores than comparison students, higher scores on Math, and higher scores on the reading and writing section on the SAT. This study was one of the largest quantitative evaluations of debate participation and achievement among high school students conducted to date and provides robust evidence of the benefits of debate on academic performance and college readiness. In a separate study where debate participants were surveyed on their views and attitudes of debate, over 96% of them “agreed” or “strongly agreed” with the statement, “Debate has enhanced my critical thinking skills.”
These attitudes indicate that respondents hold the perception that they increase their critical thinking skills through debate. Additionally, Susannah Anderson and Briana Mezuk, after reviewing data from the Chicago Debate League generated between 1997 and 2007, found that debate had a positive impact on students’ academic achievement. Some of their key findings were that among the highest risk students, 72% of debaters graduated as compared with 43% non-debaters, they scored higher on the ACT and were more likely to achieve college readiness benchmarks in English, Reading and Science.
Based on these observations, it is fair to say that competitive speech and debate helps students develop a variety of different skills that are applicable across many different segments of the education process and that there is a strong link between positive educational outcomes and participation in speech and debate. The competitive ecosystem of speech and debate allows students to become equipped with the tools that they need for school and their future job posts.
Effective use of Analytics will allow LL to best serve our customers by providing them with actionable insights that help students and parents best understand how to improve their skills to become outcome-oriented communicators.
We would like to end this portion of our paper with a quotation from Professor Alfred C. Snider that captures the spirit of our observations: “All over the world educational systems are being reorganised to emphasise active learning, critical thinking and creativity. I do not pretend to believe that debating is a magic bullet for all of the issues we face, but I do think it is a very strong candidate for something that can be done to better prepare students for the future” (xvii). We agree wholeheartedly with the late Professor Snider. Debating may not be a panacea for all challenges in the education system, though the activity is proving to be one of the most powerful methods to teach students skills required to succeed in the twenty-first century.
