本次美国代写是一个Python MDP和差分隐私的assignment

## Overview

Submit your writeup including all code and plots as a PDF via Gradescope. We recommend

reading through the entire homework beforehand and carefully using functions for testing

procedures, plotting, and running experiments. Taking the time to test, maintain, and

reuse code will help in the long run!

Data science is a collaborative activity. While you may talk with others about the

homework, please write up your solutions individually. If you discuss the homework with

your peers, please include their names on your submission. Please make sure any hand

written answers are legible, as we may deduct points otherwise.

## 1 Markov Decision Process for Robot Soccer

A soccer robot R is on a fast break toward the goal, starting in position 1. From positions

1 through 3, it can either shoot (S) or dribble the ball forward (D). From 4 it can only

shoot. If it shoots, it either scores a goal (state G) or misses (state M). If it dribbles, it

either advances a square or loses the ball, ending up in state M.

In this Markov Decision Process (MDP), the states are 1, 2, 3, 4, G, and M, where G

and M are terminal states. The transition model depends on the parameter y, which is

the probability of dribbling successfully (i.e., advancing a square). Assume a discount of

γ = 1. For k ∈ {1, 2, 3, 4}, we have

and rewards are 0 for all other transitions.

(a) (3 points) Denote by V π the value function for the specific policy π. What is V π(1)

for the policy π that always shoots?

(b) (4 points) Denote by Q∗(s, a) the value of a q-state (s, a), which is the expected

utility when starting with action a at state s, and thereafter acting optimally. What

is Q∗(3, D) in terms of y?

(c) (3 points) For what range of values of y is Q∗(3, S) ≥ Q∗(3, D)? Interpret your answer

in plain English.

**程序辅导定制C/C++/JAVA/安卓/PYTHON/留学生/PHP/APP开发/MATLAB**

本网站支持 Alipay WeChatPay PayPal等支付方式

**E-mail:** vipdue@outlook.com **微信号:**vipnxx

如果您使用手机请先保存二维码，微信识别。如果用电脑，直接掏出手机果断扫描。