| CS-2022-01 | ||||
|---|---|---|---|---|
| Title | Synthesizing Guide Programs for Sound, Effective Deep Amortized Inference | |||
| Authors | Jianlin Li, Leni Ven, Pengyuan Shi and Yizhou Zhang | |||
| Abstract |
In
probabilistic
programming
languages
(PPLs),
a
critical
step
in
optimization-based
inference
methods
is
constructing,
for
a
given
model
program,
a
trainable
guide
program.
Soundness
and
effectiveness
of
inference
rely
on
constructing
good
guides,
but
the
expressive
power
of
a
universal
PPL
poses
challenges. This paper introduces an approach to automatically generating guides for deep amortized inference in a universal PPL. Guides are generated using a type-directed translation per a novel behavioral type of system. Guide generation extracts and exploits independence structures using a syntactic approach to conditional independence, with a semantic account left to further work. Despite the control-flow expressiveness allowed by the universal PPL, generated guides are guaranteed to satisfy a critical soundness condition and, moreover, consistently improve training and inference over state-of-the-art baselines for a suite of benchmarks. | |||
| Date | December 27, 2022 | |||
| Report | CS-2022-01 (PDF) | |||