Special Topics in Immersive Media
Introduction to Motion Capture
Marker-Based and Markerless Techniques and Applications
Many aspects including the structure of this class and syllabus was inspired/ adopted/ adapted from Prof. Jordan Boyd-Graber’s NLP CMSC 470 course.
Class Location: IRB 0110 (IMD Lab)
Schedule: Thursdays, 3:30pm – 6:00pm EST
Office Hours: IRB 0110 / Online: By appointment
Course Credit: 3 Hours class, 1 day a week (3 Credits)
Course Schedule Page
Please see the schedule page for full detailed semester schedule.
General (UMD) Policies and Resources for Undergraduate Students
It is our shared responsibility to know and abide by the University of Maryland’s policies that relate to all courses.
Click here to visit for detailed policies page.
Syllabus
Course Format:
This is a flipped course. Most lectures will be delivered through video recording. In-class will be discussions, brainstorming, hands-on learning, experiments, and applying lessons learned from the video lectures.
Course Description:
This course provides an in-depth understanding of motion capture (mocap) systems, focusing on both marker-based and markerless technologies. Students will gain hands-on experience with industry-standard systems such as Vicon Shogun and Nexus for marker-based mocap, and Captury for markerless mocap, learning to effectively apply each approach based on its strengths.
The course begins with the fundamentals of motion capture, exploring how both technologies capture human movement for various industries like virtual reality (VR), augmented reality (AR), animation, gaming, and robotics. Special emphasis will be placed on understanding the strengths and limitations of both systems and selecting the appropriate technology—or combination—for different applications.
A significant component of the course will involve learning ideation and creation of a major showcase project, where students will choose to focus on one of the four categories: VR/AR, animation, gaming, or robotics. The course will culminate in a final showcase of these projects, allowing students to demonstrate their practical application of motion capture technologies.
Course Goals:
Students will understand the state-of-the-art in motion capture technologies, their strengths and limitations, and be able to evaluate their application needs to design a mocap setup that meets the needs. Students will learn how to collect, process, and apply mocap data towards various applications to get a practical understanding of challenges and experience working with mocap data and gain exposure to different kinds of applications. Finally students will design and create their own project towards a public showcase level immersive media presentation, interactive applications, video game, animation, and health and education research data analysis. Through these personal projects, students will learn end-to-end how to leverage motion capture technologies for real-life applications.
Approach:
Since this is a flipped class, most classes will begin with an unannounced quiz on the materials covered in the lectures. So you will need to watch the lecture before class and come prepared for the in-class experience.
There will be a mid-term exam that will serve as a checkpoint milestone to test a high-level understanding of concepts and your readiness for applying fundamental concepts learned towards the project.
The students will work on a final individual or a group project. The instructor will have to approve the scope and structure of the project and the team. This is to ensure an achievable project and scope is selected in three phases:
- proposal presentation, outline and clearly explaining the idea, the plan, the roles, and the scope of the project
- prototype or MVP (minimal viable project) milestone deliverable to make sure students are grappling with the problems and pivoting when necessary, and creating working prototype of the final project
- a final project demo, presentation, and/ or report.
Finally, it is required that you have regular access to a computer with sufficient capability (see min. computer requirements) and reliable internet connection throughout the course. A laptop is preferable and should be brought to class for hands on in-class activities.
Background:
Math (Highly Recommended): Basics of 3D Geometry and Linear Algebra.
Programming (Required): CMSC131 or IMDM127.
General Computer (Highly Recommended): General mastery of computer usage. You should be comfortable and fluent in either Windows or Mac or Linux daily operations.
Other Software Tools (Recommended): Unity (Game Engine), Blender (3D Computer Graphics), git (version control), Discord (Instant Messaging and VoIP).
Grading:
The grading breakdown for the course will be as follows:
| Grading Aspect | Percentage |
|---|---|
| Final Project: | 30% |
| Exam: | 30% |
| Homework Assignments: | 25% |
| Quizzes: | 5% |
| Participation: | 5% |
| Lab Etiquette: | 5% |
| Extra Credit | 5% |
Grade Cutoff:
Your cumulative score will be rounded once at the end of the semester to a single decimal place. Letter grades will be assigned as follows:
| + | 97.0% | + | 87.0% | + | 77.0% | + | 67.0% | ||
| A | 94.0% | B | 84.0% | C | 74.0% | D | 64.0% | F | < 60.0% |
| – | 90.0% | – | 80.0% | – | 70.0% | – | 60.0% |
Final Project:
The final project will need to be presented. The presentation will a showcase demo that will be open to public. Students will apply the knowledge gained to develop a project in one of four categories: virtual reality (VR)/augmented reality (AR), animation, gaming, or research data/ visualization. The goal would be to demonstrate the students mastery of motion capture technologies in a real-world context. More information will be posted on a separate page for the final project.
Final project will either be individual or a groups of two or three (max). Please note that the expectations with more number of members will be higher. While it is not practical or meaningful (in my personal opinion) to give a relative weight of effort, I will reserve the right to adjust the overall score received if the project presentation is more or less than a reasonable expectation from a 400-level IMD undergraute project.
For projects with groups, all members of the group will receive the same grade. It’s impossible for the course staff to adjudicate Rashomon-style accounts of who did what, and the goal of a group project is for all team members to work together to create a cohesive project that works well together. While it makes sense to divide the work into distinct areas of responsibility, at grading time we have now way to know who really did what, so it’s the group’s responsibility to create a piece of output that reflects well on the whole group.
Exams:
There will be one midterm. The midterm will cover material in the previous lectures and you will be allowed to use notes clearly defined by the instructor within a week before the exam.
Homework Assignments:
There will be six homework assignments. Your homework assignment with the lowest score will be dropped and not count towards your final grade. Each homework assignment is worth 5% of your final grade.
Assignments have been designed to help you learn, not just the concepts but also the nuances of the techniques, the tools, and the trade. Without practice and doing it yourself, you might get a false sense of understanding and experience. So please make sure you take full advantage of this opportunity!
You are allowed to collaborate with others (as many people as you’d like), but you must turn in your own assignment. For example, if you work together in a group, each person must write up their solutions individually. Everything you submit must be your own. Copying and pasting from any source, or making minor modifications the assignment will be considered plagiarism and will be handled according to the university’s academic integrity policies.
Due Date:
Assignments are due at 23:59 AOE, the day before the class i.e. 8 am EST on Thursdays of our class day as indicated on the schedule.
Late Policy:
Late assignments will NOT be graded. The grade of the lowest homework assignment will be dropped and not count towards your final grade.
Quizzes:
We will have unannounced quizzes throughout the semester. This is to ensure students are keeping up with the course material. The lowest quiz grade will be dropped. Because the quizzes are frequent and many of the quiz grades will be dropped, there will not be the ability to make up missed quizzes. With very rare exceptions, most students get full credit for the 5% of the grade that is made up by quizzes.
Participation:
What counts as participation? Participation includes, but not limited to, in-person attendance, polls, discussions, Q&A, and contributing to the online class forum.
Each class is critical to your learning experience, and I expect you to come to class prepared (having read all assigned readings, reviewed video lectures, and ready to engage). I also expect active participation, not passive reception of the material. Your energy in contributing to class discussions, polls, and hands-on exercises will make this class an enjoyable experience for all of us. Class participation contributes to your total grade.
We will also be using the online learning platforms. You can get credit for participation by answering and asking useful questions on that platform. Ideally you should be participating both online and in class.
Because this is a flipped class, the lectures are available online. However, the in-person class, the discussion, and lab sessions will not be recorded.
Extra Credit:
There will be opportunity to earn extra credit. Some of these include, but not limited to, specific extra credit problem in the homework assignments, a challenging/ intriguing questions that might come up during the class, impromptu extra credit challenge that the instructor might ask, helping other members of this class (for this the student being helped needs to nominate you and I will evaluate if an extra credit is deserved), or by attending relevant extra-curricular events that I will announce, if any.