MIT Open Access Repository http://repository.aust.edu.ng/xmlui/handle/123456789/659 2026-04-25T22:59:45Z 2026-04-25T22:59:45Z http://repository.aust.edu.ng/xmlui/handle/1721.1/119255 2018-11-28T10:19:48Z 2018-11-26T00:00:00Z

Gen: A General-Purpose Probabilistic Programming System with Programmable Inference Probabilistic modeling and inference are central to many fields. A key challenge for wider adoption of probabilistic programming languages is designing systems that are both flexible and performant. This paper introduces Gen, a new probabilistic programming system with novel language con- structs for modeling and for end-user customization and optimization of inference. Gen makes it practical to write probabilistic programs that solve problems from multiple fields. Gen programs can combine generative models written in Julia, neural networks written in TensorFlow, and custom inference algorithms based on an extensible library of Monte Carlo and numerical optimization techniques. This paper also presents techniques that enable Gen’s combination of flexibility and performance: (i) the generative function inter- face, an abstraction for encapsulating probabilistic and/or differentiable computations; (ii) domain-specific languages with custom compilers that strike different flexibility/per- formance tradeoffs; (iii) combinators that encode common patterns of conditional independence and repeated compu- tation, enabling speedups from caching; and (iv) a standard inference library that supports custom proposal distributions also written as programs in Gen. This paper shows that Gen outperforms state-of-the-art probabilistic programming systems, sometimes by multiple orders of magnitude, on problems such as nonlinear state-space modeling, structure learning for real-world time series data, robust regression, and 3D body pose estimation from depth images.

2018-11-26T00:00:00Z http://repository.aust.edu.ng/xmlui/handle/1721.1/118307 2018-11-26T22:27:54Z 2018-10-01T00:00:00Z

Towards Understanding Generalization via Analytical Learning Theory This paper introduces a novel measure-theoretic theory for machine learning that does not require statistical assumptions. Based on this theory, a new regularization method in deep learning is derived and shown to outperform previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed theory provides a theoretical basis for a family of practically successful regularization methods in deep learning. We discuss several consequences of our results on one-shot learning, representation learning, deep learning, and curriculum learning. Unlike statistical learning theory, the proposed learning theory analyzes each problem instance individually via measure theory, rather than a set of problem instances via statistics. As a result, it provides different types of results and insights when compared to statistical learning theory.

2018-10-01T00:00:00Z http://repository.aust.edu.ng/xmlui/handle/1721.1/118184 2018-11-26T22:27:53Z 2018-09-27T00:00:00Z

Using Dynamic Monitoring to Synthesize Models of Applications That Access Databases We previously developed Konure, a tool that uses active learning to infer the functionality of database applications. An alternative approach is to observe the inputs, outputs, and database traffic from a running system in normal use and then synthesize a model of the application from this information. To evaluate these two approaches, we present Etch, which uses information from typical usage scenarios to synthesize a model of the functionality of database applications whose computation can be expressed in the Konure DSL.

2018-09-27T00:00:00Z http://repository.aust.edu.ng/xmlui/handle/1721.1/117593 2018-11-26T22:27:53Z 2018-08-28T00:00:00Z

Using Active Learning to Synthesize Models of Applications That Access Databases We present a new technique that uses active learning to infer models of applications that manipulate relational databases. This technique comprises a domain-specific language for modeling applications that access databases (each model is a program in this language) and an associated inference algorithm that infers models of applications whose behavior can be expressed in this language. The inference algorithm generates test inputs and database configurations, runs the application, then observes the resulting database traffic and outputs to progressively refine its current model hypothesis. The end result is a model that completely captures the behavior of the application. Because the technique works only with the externally observable inputs, outputs, and databases, it can infer the behavior of applications written in arbitrary languages using arbitrary coding styles (as long as the behavior of the application is expressible in the domain-specific language). We also present a technique for automatically regenerating an implementation from the inferred model. The regenerator can produce a translated implementation in a different language and systematically include relevant security and error checks.

2018-08-28T00:00:00Z