With the influx of EdTech that has appeared on the market in recent years, colleges and universities are becoming more selective about which EdTech products they choose for their students–and are prioritizing companies that show that their product works. For EdTech companies, having research-backed evidence of the effectiveness of their product is now an integral part of their company’s success.
But, how do you get started with research? Although many people may equate ‘research’ with ‘randomized-control trial’ (RCT) – oftentimes considered the ‘gold standard’ of research – it is unlikely that early stage EdTech products will be ready to conduct an RCT.
When working with EdTech startups in our Accelerator at WGU Labs, we collaboratively assess a company’s “research readiness” to determine what type of research study we can conduct. Not every product is ready for or even needs an RCT, but all companies can integrate research into their design and development process to gather evidence of the impact of their product.
To determine your EdTech product’s research readiness, consider the following three questions:
1 – What Stage Is Your Product In?
Are you about to implement your product in its very first classroom? Or is your product already being used at multiple institutions? Are you still working on a prototype of your product? Or is it ready to be delivered to students? Determining what stage your product is in will help determine the most appropriate research study design to evaluate its impact.
If your product is still in its early stages, and you have not yet considered the different learning theories that play out when users interact with your product, you might consider creating a logic model. A logic model maps out the activities involved and how they impact your outcomes of interest. By outlining the outputs of your product and how users reach these endpoints, you may have a better sense of missing features or potential areas of improvement.
Or, if your product is at a later stage where it is ready to be implemented, it might be most useful to conduct an implementation study, which may help assess and troubleshoot any difficulties that users have when interacting with your product in real education environments, thus improving the experience for future implementations.
And, if your product has proven usability in education environments, only then should formal survey research, longitudinal studies, and potentially RCTs be considered.
2 – Have You Secured Research Sites and Samples?
Before committing to any research, it is crucial that you have a research site with real users that you can include in conversations around the research.
Collaborative research sites are especially important if you are looking to include institutional data in your research. Imagine the frustration of writing up a research plan only to find that the research site you end up with does not want to implement certain research designs or share certain data you had included in your plan. This is why it is imperative to include your research site at the beginning of the research design process.
3 – How Are You Defining Impact and Success With Your Product?
What are the outcomes you are interested in assessing? Expand your thinking outside the traditional, and normative, outcomes like course grades and grade point average. Although these are certainly important success metrics, it is also important to consider how your product impacts students’ learning experiences and the crucial learning skills that aren’t necessarily reflective in grades alone. For example, when designing research with our partner Peerceptiv, we assessed the effect of peer feedback on course outcomes, but also assessed how use of this product impacted peer collaboration, a crucial 21st century learning skill.
After coming up with the outcomes of interest, be certain you know how you are going to collect these outcomes. Surveys? Interviews? Institutional data? Make sure your team has the capabilities to deliver these research instruments, as well as have access to institutional data. By carefully considering the outcomes that can speak to how your product impacts the learning experience from a holistic standpoint and assessing the best way of collecting this data, you will have a more powerful understanding of the role your product plays in the classroom.
The demands on EdTech products to demonstrate efficacy in real education environments are greater than ever, especially with the increasing push in higher education for research to play a bigger role in the EdTech decision-making process. Unlike K12 where evidence-based decision-making is more prevalent because of policies like The No Child Left Behind Act and its successor, the Every Student Succeeds Act (ESSA), there are no similar guidelines for higher education. Higher education folks must intentionally seek out products that are conducting rigorous research on their own. To help, EdTech developers can use our three questions to prompt preliminary thinking on the starting blocks of a research project, to guide teams thoughtfully through design, implementation, and outcomes that will result in more impactful EdTech evaluation research.
The Accelerator at WGU Labs leverages the talent of 30-plus education and business professionals to boost your company’s impact and business scale. Interested in learning more? Let’s chat.
