Created

Sep 2, 2021 06:34 PM

Topics

Components of a Learning SystemExample: Credit Approval SystemA Simple Learning ModelWeight w(t)ErrorsHypothesis Space Complexity of Goal of Machine LearningLearning ParadigmsSupervised LearningOnline LearningActive LearningUnsupervised LearningApplicationsSemi-Supervised LearningApplicationsSelf-Supervised LearningWhy WorksApplicationsReinforcement LearningCourse OverviewTODO: Homework 0

## Components of a Learning System

### Example: Credit Approval System

Input: Customer Information (demographics, income, age, gender)

Output: Binary credit decision (made money or not)

Unknow target function (Idea credit approval formula)

Training Data (histoical records of current customers)

- typically I.I.D (independent & identically distributed)

Learning Algorithm → Final Hypothesis (Learned credit approval func)

Hypothesis Set (Condidate predictive functions)

## A Simple Learning Model

**Hypothesis Set**: All linear functions

#### Weight w(t)

Update:

Assuming data linearly seperable: this algoritm h(x) works

### Errors

out-of-sample error (unknown)

in-sample error (training error - known)

### Hypothesis Space

The set of functions that includes g that best approximates f

Picking a hypothesis space = selecting the type of algo / modal to use

#### Complexity of

**Complex****: better chance of approximating in-sample**

**Simple****: better chance of generalizing out-of-sample**

### Goal of Machine Learning

Minimize given

## Learning Paradigms

### Supervised Learning

Explicitly provided w/ labeled inputs

#### Online Learning

Does not have access to all data upfront

Given the algo one example at a time

e.g. auto driving

#### Active Learning

Algo allowed to query egs for its training

### Unsupervised Learning

Only given input examples w/o annotations

- Like human's obsersational learning

- A hard problem

#### Applications

- Clustering Images

- Embedding Images: give each images an coord to put similar ones together

- Embedding Words

### Semi-Supervised Learning

Only given a subset of labels

#### Applications

Label propagation

### Self-Supervised Learning

Infer labels from large collections of unlabeled data by exploiting other properties associated w/ dataset

- Professor Worked on it during PHD

#### Why Works

Supervised: hitting its limit

Unsupervised: too hard to approach

#### Applications

Exploiting temporal correlatons of video frames to deduce that image from nearby frames are similar

### Reinforcement Learning

Agent finds its own way & use positive/negative reward/punishment for guide

## Course Overview

- ML Overview: learning & generalization

- Probabilistic approach

- Linear Parametric models

- Non-Linear Parametric models

- Unsupervised

## TODO: Homework 0

**Content**: questions from probability, calculus, linear algebra (10 questions)

On Brightspace

‣