You'll start by applying computer vision and deep learning to automotive problems like detecting lane lines and predicting steering angles. Following that, you'll learn sensor fusion, which will allow you to filter data from an array of sensors in order to perceive the environment. Finally, you'll be able to test your code in a simulation on Udacity's self-driving car.
Course 1: Introduction
LESSON ONE Welcome
- Take your first steps towards becoming a Self-Driving Car Engineer! In this lesson, we’ll introduce you to the program, help you discover the services we provide, and show you all the incredible projects you’ll build. Get ready for an incredible 6-month journey!
LESSON TWO Workspaces
- Many projects and some quizzes will be accessed via Workspaces. These workspaces streamline environment setup, simplify project submission, and can be enabled with GPU support. All workspaces are Linux-based and can be interfaced via a shell (BASH). Some workspace interfaces are direct from the shell, others run a JUPYTER Notebook server and interaction is mainly through the JUPYTER notebook interface.
Course 2: Computer Vision
LESSON ONE Computer Vision Fundamentals
- In this first lesson, you’ll get taste of some basic computer vision techniques to find lane markings on the road. We will be diving much deeper into computer vision in later lessons, so just relax and have some fun in this first week!
LESSON TWO Advanced Computer Vision
- Discover more advanced computer vision techniques, like distortion correction and gradient thresholding, to improve upon your lane lines algorithm!
Course Project Finding Lane Lines on a Road
In this project, you will write code to recognise lane lines on the road, first in an image and then in a video stream (which is really just a series of images). To complete this project, you will build on the tools you learned about in the lesson.
Course Project Advanced Lane Finding
The goal of this project is to write a software pipeline to identify lane boundaries in a video captured by a front-facing camera on a car.
Course 3: Deep Learning
LESSON ONE Neural Networks
- Learn to build and train neural networks, starting with the foundations in linear and logistic regression, and culminating in backpropagation and multilayer perceptron networks.
LESSON TWO TensorFlow
- Vincent Vanhoucke, Principal Scientist at Google Brain, introduces you to deep learning and Tensorflow, Google’s deep learning framework.
LESSON THREE Deep Neural Networks
- Vincent walks you through how to go from a simple neural network to a deep neural network. You’ll learn about why additional layers can help and how to prevent overfitting.
LESSON FOUR Convolutional Neural Networks
- Vincent explains the theory behind Convolutional Neural Networks and how they help us dramatically improve performance in image classification.
LESSON FIVE Keras
- Take on the neural network framework, Keras. You’ll be amazed how few lines of code you’ll need to build and train deep neural networks!
LESSON SIX Transfer Learning
- Learn about some of the most famous neural network architectures and how you can use them. By the end of this lesson, you’ll know how to create new models by leveraging existing canonical networks.
Course Project Traffic Sign Classifier
You've just gotten your feet wet in the world of deep learning. Put your knowledge to the test by using deep learning to classify various traffic signs! You will use what you've learned about deep neural networks and convolutional neural networks to classify traffic signs in this project.
Course Project Behavioral Cloning
With this project, you can put your deep learning skills to the test! Train a deep neural network to drive your car!
Course 4: Sensor Fusion
LESSON ONE Sensors
- Meet the team at Mercedes who will help you track objects in real-time with Sensor Fusion.
LESSON TWO Kalman Filters
- Learn from the best! Sebastian Thrun will walk you through the usage and concepts of a Kalman Filter using Python.
LESSON THREE C++
- Checkpoint Are you ready to build Kalman Filters with C++? Take these quizzes to find out.
LESSON FOUR Extended Kalman Filters
- In this lesson, you’ll build a Kalman Filter in C++ that’s capable of handling data from multiple sources. Why C++? Its performance enables the application of object tracking with a Kalman Filter in real-time.
Course Project Extended Kalman Filte
In this project, you will put everything you've learned about Sensor Fusion to use by creating an Extended Kalman Filter in C++!
Course 5: Localization
LESSON ONE Introduction to Localization
- Meet the team that will guide you through the localization lessons!
LESSON TWO Markov Localization
- In this lesson, you’ll learn the math behind localization as well as how to implement Markov localization in C++.
LESSON THREE Motion Models
- Here you’ll learn about vehicle movement and motion models to predict where your car will be at a future time.
LESSON FOUR Particle Filters
- Sebastian will teach you what a particle filter is as well as the theory and math behind the filter.
LESSON FIVE Implementation of a Particle Filter
- Now that you understand how a particle filter works, you’ll learn how to code a particle filter.
Course Project Kidnapped Vehicle
In this project, you will create a particle filter and use it in conjunction with a real map to locate a vehicle!
Course 6: Path Planning
LESSON ONE Search
- In this lesson you will learn about discrete path planning and algorithms for solving the path planning problem.
LESSON TWO Prediction
- In this lesson you’ll learn how to use data from sensor fusion to generate predictions about the likely behavior of moving objects.
LESSON THREE Behavior Planning
- In this lesson you’ll learn how to think about high level behavior planning in a self-driving car.
LESSON FOUR Trajectory Generation
- In this lesson, you’ll use C++ and the Eigen linear algebra library to build candidate trajectories for the vehicle to follow.
Course Project Highway Driving
Drive a car down a highway with other cars using your own path planner.
Course 7: Control
LESSON ONE PID Control
- In this lesson you’ll learn about and how to use PID controllers with Sebastian!
Course Project PID Controller
You will revisit the lake race track from the Behavioral Cloning Project in this project. This time, however, you will use C++ to implement a PID controller to guide the vehicle around the track!
Course Project Build Your Online Presence Using LinkedIn
LinkedIn is a great place to showcase your portfolio and get feedback.
Course Project Optimize Your GitHub Profile
Create a GitHub profile that rivals that of senior software engineers.
Course 8: System Integration
LESSON ONE Autonomous Vehicle Architecture
- Learn about the system architecture for Carla, Udacity’s autonomous vehicle.
LESSON TWO Introduction to ROS
- Obtain an architectural overview of the Robot Operating System Framework and setup your own ROS environment on your computer.
LESSON THREE Packages and Catkin Workspaces
- Learn about ROS workspace structure, essential command line utilities, and how to manage software packages within a project. Harnessing these will be key to building shippable software using ROS.
LESSON FOUR Writing ROS Nodes
- ROS Nodes are a key abstraction that allows a robot system to be built modularly. In this lesson, you’ll learn how to write them using Python.
Course Project System Integration
You will run your code on a virtual version of Carla, Udacity's self-driving car, in this project.