Blade-2, GPT-4 or Claude-2;  Which AI language model is the best?

Can computer vision systems infer your muscle activity from video? Meet Muscles in Action (MIA): A New Dataset for Learning to Incorporate Muscle Activity into Representations of Human Movement

In recent times, the field of Artificial Intelligence has been under discussion. Whether it is the human-mimicking Large Language Model like GPT 3.5 based on natural language processing and natural language understanding or the text-in-image model called DALL-E based on computer vision, AI is paving the way to success. Computer vision, the subfield of AI, is getting better with each new application being released. It became able to analyze human movement from video and thus tackle various tasks such as pose estimation, action recognition and motion transfer.

While computer vision has advanced in determining human movement, it’s not just about the outward appearance. Every action is the consequence of our brain transmitting electrical impulses to our nerves, which in turn cause our muscles to contract, resulting in joint movement. Researchers have worked hard to develop an approach with the help of which it is possible to simulate the intrinsic muscle activity that drives human mobility. To advance this research, two Columbia University researchers have introduced a new and unique data set called “Muscles in Action” (MIA). This dataset includes 12.5 hours of synchronized video and surface electromyography (sEMG) data and captures ten subjects performing various exercises.

Surface electromyography (sEMG) sensors, available in invasive and non-invasive versions, are the traditional tool for determining muscle activity. Researchers developed a representation that could predict muscle activation from video and, in the opposite direction, reconstruct human movement from muscle activation data using the MIA dataset. The main goal is to understand the complex connection between underlying muscle activity and visual information. By jointly modeling both modalities, the model was conditioned to generate motion consistent with muscle activation.

Build high-quality training datasets with Kili Technology and solve NLP machine learning challenges to develop powerful ML applications

The main part of this project is the framework for modeling the link between the human movement seen in the video and the internal muscle activity reflected by sEMG signals. The research paper shared by the team offers a brief overview of relevant work in human activity analysis, conditional motion generation, multimodal learning, electromyography, and physics-based human motion generation. An in-depth description and analysis of the multimodal data set follows.

For the evaluation, the researchers experimented with both the participants and the exercises in the distribution and the subjects and the exercises out of the distribution to determine how well their model worked. They tested the model on data other than the training distribution and data similar to the data it was trained on. This evaluation helps validate the generalizability of the methodology.

In conclusion, the use of muscles in computer vision systems has numerous potential uses. Richer virtual human models can be produced by understanding and simulating internal muscle action. These models can be used in a variety of real-world contexts, including those related to sports, fitness, augmented reality, and virtual reality.


Check out thePaper ANDProject page.All the credit for this research goes to the researchers of this project. Also, don’t forget to subscribeour 26k+ ML SubReddit,Discord channel,ANDEmail newsletterwhere we share the latest news on AI research, cool AI projects and more.


Tanya Malhotra is a final year student at Petroleum and Energy University, Dehradun pursuing BTech in Computer Engineering with a major in Artificial Intelligence and Machine Learning.
She is a data science enthusiast with good analytical and critical thinking, coupled with a burning interest in acquiring new skills, leading teams, and managing work in an organized manner.


Gain a competitive edge with data – actionable market insights for global brands, retailers, analysts and investors. (Sponsored)

#computer #vision #systems #infer #muscle #activity #video #Meet #Muscles #Action #MIA #Dataset #Learning #Incorporate #Muscle #Activity #Representations #Human #Movement
Image Source : www.marktechpost.com

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *