• Basic information
• Learning outcomes
• Contents
• Assessment
• Workload

Learning outcomes

Upon successful completion of this course, students will be able to:

1. Each student is expected to write a report in the form of a paper as part of the project. This approach contributes to the development of their writing skills.

   Contribution to Program Outcomes

   1. To gain in-depth knowledge about the sensor systems utilized in military applications.
   2. To enroll in and contribute to the R&D projects.
   3. Design and conduct independent research projects.
   4. Develop an awareness of continuous learning in relation with modern technology
   5. Find out new ways to improve current knowledge
   6. Write progress reports based on published documents, dissertations, articles.
   7. Demonstrating professional and ethical responsibility.
   8. Be aware of the importance of nanoscience and nanoengineering in understanding the working principles of the new generation nano devices.

   Method of assessment

   1. Written exam
   2. Homework assignment
   3. Seminar/presentation
   4. Term paper
2. The goal is to develop the algorithmic skills needed for building self-learning, autonomous systems.

   Contribution to Program Outcomes

   1. To gain insight and experience about the solution approaches to the technical problems encountered in projects.
   2. To enroll in and contribute to the R&D projects.
   3. To understand the basic principles and applications of new tools and / or software required for thesis work.
   4. Write progress reports based on published documents, dissertations, articles.
   5. Present and defence the research outcomes at seminars and conferences
   6. Be aware of the importance of nanoscience and nanoengineering in understanding the working principles of the new generation nano devices.

   Method of assessment

   1. Written exam
   2. Homework assignment
   3. Seminar/presentation
   4. Term paper

Contents
Week 1:	Fundamental topics in machine learning and reinforcement learning. Introduction to programming language and packages to be used in the course.
Week 2:	Introduction to reinforcement learning.
Week 3:	Tabular methods. Markov decision processes. Monte Carlo methods. Dynamic programming. Bellman equations.
Week 4:	Temporal difference method. TD(0). Sarsa.
Week 5:	Q-learning.
Week 6:	Approximation functions. Gradient descent.
Week 7:	Artificial neural networks. Using neural networks in reinforcement learning.
Week 8:	Policy gradient approaches. REINFORCE. Actor-critic methods.
Week 9:	DQN and variants.
Week 10:	A2C, A3C, DDPG algorithms.
Week 11:	Solving sample problems in reinforcement learning.
Week 12:	Solving sample problems in reinforcement learning.
Week 13:	Solving sample problems in reinforcement learning.
Week 14:	Solving sample problems in reinforcement learning.
Week 15*:	Solving sample problems in reinforcement learning.
Week 16*:	project presentation
Textbooks and materials:	Reinforcement Learning: An Introduction, Suton, Barto, 2015.
Recommended readings:	Applied Text Analysis with Python, Bengfort, Bilbro, Ojeda, 2018.
	* Between 15th and 16th weeks is there a free week for students to prepare for final exam.

Assessment
Method of assessment	Week number	Weight (%)
Mid-terms:	7	30
Other in-term studies:		0
Project:	1	20
Homework:	5	30
Quiz:		0
Final exam:	14	20
	Total weight:	(%)

Workload
Activity	Duration (Hours per week)	Total number of weeks	Total hours in term
Courses (Face-to-face teaching):	3	16
Own studies outside class:	4	16
Practice, Recitation:	0	0
Homework:	2	5
Term project:	2	2
Term project presentation:	2	2
Quiz:	0	0
Own study for mid-term exam:	3	8
Mid-term:	2	2
Personal studies for final exam:	3	8
Final exam:	2	2
		Total workload:
		Total ECTS credits:	*
	* ECTS credit is calculated by dividing total workload by 25. (1 ECTS = 25 work hours)