I wonder if any of you are like me: completely clueless about physics, with no understanding of why circuits, classical mechanics, optics, and electromagnetism work the way they do. Recently, an overseas team published a paper on a tool that can create embedded interactive physics simulations in textbook diagrams. The tool is called Augmented Physics. The research team states that Augmented Physics can extract images or tables from physics textbooks and generate simulated interactions based on the extracted content, promoting interactive and personalized learning experiences for users regarding physics concepts such as optics, circuits, kinematics, and more.
Physics is no longer hard to understand! Augmented Physics: Turning static physics diagrams into vivid interactive simulations.
Recently, at the 2024 User Interface Software and Technology Symposium (UIST ’24), a paper titled “Augmented Physics: Creating Interactive Embedded Physics Simulations from Static Textbook Diagrams” appeared. Augmented Physics uses image segmentation models (SAM) and multimodal large language models (Multi-modal LLMs) to allow users to semi-automatically extract images or tables from physics textbooks, and it generates interactive simulations based on the extracted content. These interactive diagrams are then placed back onto the scanned textbook pages, facilitating users’ understanding of various physics concepts.
If the introduction above isn’t clear enough, you can think of Augmented Physics as a tool that turns static images into dynamic ones. For example, when you look at a circuit diagram, someone who isn’t very familiar with physics might not be able to picture how the image actually works. Augmented Physics can help by simulating how the image operates in practice, strengthening your understanding of the circuit diagram so you no longer have to rely only on your imagination to grasp how physics works.
Of course, Augmented Physics can be used not only for circuit diagrams but also for kinematic images. For example, in diagrams related to Newton’s laws of motion, you can segment various objects—isolating the slope as a static object and the skier as a dynamic object—so that Augmented Physics can simulate the skier’s motion trajectory.
Augmented Physics can also bind the question’s data and images in both directions (Bi-directional Binding), allowing users to observe in real time the changes brought about by different data.
Augmented Physics can also visualize various parameters generated from simulation charts according to user needs. Simply put, with Augmented Physics, you no longer need to think about physics problems abstractly.
Currently, Augmented Physics is just a paper, though it did win the Best Paper Award at the 2024 ACM Symposium on User Interface Software and Technology (UIST ’24). It’s hard to say whether this tool will actually be developed in the future. Technology really is advancing. Back when I was in middle school, I couldn’t visualize physics problems in my head and couldn’t understand why certain physical motions happened the way they did. If I’d had something like this back then, I probably wouldn’t have developed a fear of physics, and it might even have made me more passionate about learning it. I hope Augmented Physics can really be developed in the future to help others who, like me, are working hard to learn and understand physics.
If you’re interested in learning more about the Augmented Physics tool, you can watch the video below orGo to the Augmented Physics paper website.Learn more:
Source: KOCPC Chinese