ELEC3544 · Fall 2026

Data Science with Foundation Models

Interactive and efficient data science powered by modern agentic AI

This course develops the foundations and practical skills for turning data into reliable insights and predictive models. It combines mathematical intuition, statistical reasoning, Python demonstrations, and applications. Foundation models and AI agents support the workflow throughout, while students learn to verify generated code, assess evidence, and explain their decisions.

HKU Wednesdays · 4:00–6:50 PM CPD-1.21

Course staff

Meet the teaching team

Course calendar

Nine lectures, one midterm, and two project-presentation sessions.

Date Topic and focus Materials
Sep 2
Week 1 · L1
Introduction to Data Science and Foundation Models
Data science workflows, data collection, exploratory analysis, visualization, AI prompting, and verification.
Slides (PDF)
5.3 MB
Sep 9
Week 2 · L2
Linear Algebra for Data Science
Data matrices, similarity, covariance, eigenvectors, principal component analysis, and explained variance.
Slides (PDF)
5.6 MB
Sep 16
Week 3 · L3
Probability and Statistics for Data Science
Distributions, conditional probability, Bayes’ rule, confidence intervals, hypothesis testing, and p-values.
Slides (PDF)
4.2 MB
Sep 23
Week 4 · L4
Data Science Applications and Data Wrangling
Problem formulation, data quality, missing data, cleaning, feature encoding, scaling, and reproducible preprocessing.
Slides (PDF)
5.2 MB
Sep 30
Week 5 · L5
Regression: Predicting Continuous Outcomes
Linear regression, coefficient interpretation, model evaluation, overfitting, the bias–variance tradeoff, and regularization.
Slides (PDF)
6.0 MB
Oct 7
Week 6 · L6
Classification: Predicting Categories
Logistic regression, decision trees, thresholds, precision and recall, ROC-AUC, and class imbalance.
Slides (PDF)
5.0 MB
Oct 14
Week 7
Reading week
No scheduled lecture.
—
Oct 21
Week 8 · Exam
Midterm exam
In class; closed book.
—
Oct 28
Week 9 · L7
Unsupervised Learning and Embeddings
Polynomial features, k-means clustering, learned embeddings, cosine similarity, semantic search, and transfer.
Slides (PDF)
3.8 MB
Nov 4
Week 10 · L8
Deep Learning
Neural networks and function approximation, backpropagation, automatic differentiation, optimizers, CNNs, and ResNet.
Slides (PDF)
11.3 MB
Nov 11
Week 11 · L9
Foundation Models and Agentic Data Science (planned)
Attention, pretraining, LLMs, and tool-using agents for data science workflows, with verification and evaluation.
Slides (PDF)
9.0 MB
Nov 18
Week 12 · P1
Group project presentations
First presentation session; approximately 10 minutes per group.
—
Nov 25
Week 13 · P2
Group project presentations
Second presentation session; approximately 10 minutes per group.
—

Scroll the schedule sideways on small screens to see the materials.

Course structure

Lectures Focus Topics
L1–L4 Foundations The data science workflow, linear algebra, probability and statistics, applications, data quality, and preprocessing.
L5–L7 Learning from data Regression, classification, unsupervised learning, and embeddings, with emphasis on evaluation and generalization.
L8–L9 Modern AI Deep learning, foundation models, LLMs, and agentic systems for interactive data science applications.

Learning outcomes

By the end of the course, students should be able to:

  • Formulate a data science problem and build a reproducible analysis workflow.
  • Prepare and visualize data, and interpret uncertainty and statistical evidence.
  • Select, fit, and evaluate supervised and unsupervised models on unseen data.
  • Explain learned representations and the core mechanisms of deep learning and foundation models.
  • Use AI agents effectively, verify their outputs, and communicate and defend the results.

Assessment

Weight Component Details
10% Attendance and participation Starting from Week 2.
35% Homework assignments Approximately every two weeks; 3–4 assignments in total.
25% Midterm exam October 21, 2026, in class. Closed book; approximately 1.5–2 hours. One A4 cheat sheet and a calculator permitted.
30% Group project Three members per group. Collaborate with AI agents and evaluate the resulting work. Report and presentations in November.

Group project milestones

  • Report deadline: November 17, 2026, at 11:59 PM.
  • Presentations: November 18 and November 25, approximately 10 minutes per group.

Project resources:

Acknowledgments

This course was originally created and taught by Prof. Yanchao Yang from 2024 through Spring 2026.

The modern AI portions draw extensively on materials from MIT 6.7960 Deep Learning, Caltech EE/CS 148B Large Language & Vision Models, and Stanford CS146S The Modern Software Developer.