The Weighted Tsetlin Machine: Compressed Representations with Weighted Clauses

by   Adrian Phoulady, et al.

The Tsetlin Machine (TM) is an interpretable mechanism for pattern recognition that constructs conjunctive clauses from data. The clauses capture frequent patterns with high discriminating power, providing increasing expression power with each additional clause. However, the resulting accuracy gain comes at the cost of linear growth in computation time and memory usage. In this paper, we present the Weighted Tsetlin Machine (WTM), which reduces computation time and memory usage by weighting the clauses. Real-valued weighting allows one clause to replace multiple and supports fine-tuning the impact of each clause. Our novel scheme simultaneously learns both the composition of the clauses and their weights. Furthermore, we increase training efficiency by replacing k Bernoulli trials of success probability p with a uniform sample of average size p k, the size drawn from a binomial distribution. In our empirical evaluation, the WTM achieved the same accuracy as the TM on MNIST, IMDb, and Connect-4, requiring only 1/4, 1/3, and 1/50 of the clauses, respectively. With the same number of clauses, the WTM outperformed the TM, obtaining peak test accuracies of respectively 98.58%, 90.15%, and 87.49%. Finally, our novel sampling scheme reduced sample generation time by a factor of 7.


page 1

page 2

page 3

page 4


A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern Representation

The Regression Tsetlin Machine (RTM) addresses the lack of interpretabil...

Non-uniform quantization with linear average-case computation time

A new method for binning a set of n data values into a set of m bins for...

AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning

With the rapid adoption of machine learning (ML), a number of domains no...

Effects of Archive Size on Computation Time and Solution Quality for Multi-Objective Optimization

An unbounded external archive has been used to store all nondominated so...

Efficient RLHF: Reducing the Memory Usage of PPO

Reinforcement Learning with Human Feedback (RLHF) has revolutionized lan...

Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability

Despite significant effort, building models that are both interpretable ...

A New Weighting Scheme in Weighted Markov Model for Predicting the Probability of Drought Episodes

Drought is a complex stochastic natural hazard caused by prolonged short...

Please sign up or login with your details

Forgot password? Click here to reset