how about MDP or POMDP

[Leon Hu]

can Figaro sovle MDP or POMDP problem ?

[Brian Ruttenberg]

Hi, Currently, not really. We can represent an MDP or POMDP, but it doesn’t contain any of the typical value or policy iteration algorithms uses to solve them. One could probably write them though as a Figaro algorithm. Brian + + + + + + + + + + + + + + Brian Ruttenberg, PhD Senior Scientist Decision Management Systems Division Government Services Charles River Analytics Inc. 617.491.3474 x730 www.cra.com<http://www.cra.com/> From: gdhucoder [mailto:[e-mail]] Sent: Wednesday, October 18, 2017 11:41 AM To: p2t2/figaro <[e-mail]> Cc: Subscribed <[e-mail]> Subject: [p2t2/figaro] how about MDP or POMDP (#725) can Figaro sovle MDP or POMDP problem ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#725>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFJOJd_nwcNnzLLHt8xdrXl3_s_HKndsks5sthv4gaJpZM4P97Ac>.

[Leon Hu]

Hi, Dear Brian,

Thanks for your kind reply! I'm just beginning to do a little research on MDP, and trying to combine probabilistic programming with dynamic planners. I found Figaro is a very nice PPS.
Your previous work is very meaningful! And I love Avi Pfeffer's book.

Yours gdHu

[apfeffer]

Hi gdHu, In a previous probabilistic programming language, IBAL, I actually showed that value iteration for MDPs is a generalization of the algorithm IBAL used. Figaro’s structured variable elimination algorithm is similar to IBAL’s algorithm. I think it’s possible, if you encode the problem in the right way, to express an MDP in Figaro, and recover the value iteration algorithm. But this is not something we have done explicitly. I’m glad you like my book. Thanks, Avi From: gdhucoder <[e-mail]> Reply-To: p2t2/figaro <[e-mail]> Date: Saturday, October 21, 2017 at 1:56 AM To: p2t2/figaro <[e-mail]> Cc: Subscribed <[e-mail]> Subject: Re: [p2t2/figaro] how about MDP or POMDP (#725) Hi, Dear Brian, Thanks for you kind reply! I'm just beginning to do a little research on MDP, and trying to combine probabilistic programming with dynamic planners. I found Figaro is a very nice PPS. Your previous work is very meaningful! And I love Avi Pfeffer's book. Yours gdHu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#725 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFJkd7SpDkghGXuva4lCGxUpIFvVemxYks5suYdvgaJpZM4P97Ac>.