Computer Science and Artificial Intelligence Lab (CSAIL)
http://repository.aust.edu.ng/xmlui/handle/123456789/660
2024-03-28T22:08:41ZGen: A General-Purpose Probabilistic Programming System with Programmable Inference
http://repository.aust.edu.ng/xmlui/handle/1721.1/119255
Gen: A General-Purpose Probabilistic Programming System with Programmable Inference
Probabilistic modeling and inference are central to many fields. A key challenge for wider adoption of probabilistic programming languages is designing systems that are both flexible and performant. This paper introduces Gen, a new probabilistic programming system with novel language con- structs for modeling and for end-user customization and optimization of inference. Gen makes it practical to write probabilistic programs that solve problems from multiple fields. Gen programs can combine generative models written in Julia, neural networks written in TensorFlow, and custom inference algorithms based on an extensible library of Monte Carlo and numerical optimization techniques. This paper also presents techniques that enable Gen’s combination of flexibility and performance: (i) the generative function inter- face, an abstraction for encapsulating probabilistic and/or differentiable computations; (ii) domain-specific languages with custom compilers that strike different flexibility/per- formance tradeoffs; (iii) combinators that encode common patterns of conditional independence and repeated compu- tation, enabling speedups from caching; and (iv) a standard inference library that supports custom proposal distributions also written as programs in Gen. This paper shows that Gen outperforms state-of-the-art probabilistic programming systems, sometimes by multiple orders of magnitude, on problems such as nonlinear state-space modeling, structure learning for real-world time series data, robust regression, and 3D body pose estimation from depth images.
2018-11-26T00:00:00ZTowards Understanding Generalization via Analytical Learning Theory
http://repository.aust.edu.ng/xmlui/handle/1721.1/118307
Towards Understanding Generalization via Analytical Learning Theory
This paper introduces a novel measure-theoretic theory for machine learning
that does not require statistical assumptions. Based on this theory, a new
regularization method in deep learning is derived and shown to outperform
previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed
theory provides a theoretical basis for a family of practically successful
regularization methods in deep learning. We discuss several consequences of
our results on one-shot learning, representation learning, deep learning,
and curriculum learning. Unlike statistical learning theory, the proposed
learning theory analyzes each problem instance individually via measure
theory, rather than a set of problem instances via statistics. As a result,
it provides different types of results and insights when compared to
statistical learning theory.
2018-10-01T00:00:00ZUsing Dynamic Monitoring to Synthesize Models of Applications That Access Databases
http://repository.aust.edu.ng/xmlui/handle/1721.1/118184
Using Dynamic Monitoring to Synthesize Models of Applications That Access Databases
We previously developed Konure, a tool that uses active learning to
infer the functionality of database applications. An alternative
approach is to observe the inputs, outputs, and database traffic from a
running system in normal use and then synthesize a model of the
application from this information. To evaluate these two approaches, we
present Etch, which uses information from typical usage scenarios to
synthesize a model of the functionality of database applications whose
computation can be expressed in the Konure DSL.
2018-09-27T00:00:00ZUsing Active Learning to Synthesize Models of Applications That Access Databases
http://repository.aust.edu.ng/xmlui/handle/1721.1/117593
Using Active Learning to Synthesize Models of Applications That Access Databases
We present a new technique that uses active learning to infer models of
applications that manipulate relational databases. This technique
comprises a domain-specific language for modeling applications that
access databases (each model is a program in this language) and an
associated inference algorithm that infers models of applications whose
behavior can be expressed in this language. The inference algorithm
generates test inputs and database configurations, runs the application,
then observes the resulting database traffic and outputs to
progressively refine its current model hypothesis. The end result is a
model that completely captures the behavior of the application. Because
the technique works only with the externally observable inputs, outputs,
and databases, it can infer the behavior of applications written in
arbitrary languages using arbitrary coding styles (as long as the
behavior of the application is expressible in the domain-specific language).
We also present a technique for automatically regenerating an
implementation from the inferred model. The regenerator can produce a
translated implementation in a different language and systematically
include relevant security and error checks.
2018-08-28T00:00:00Z