Universal Java Matrix Package

A Java Matrix Library for Linear Algebra and High Performance Computations

About

The Universal Java Matrix Package (UJMP) is an open source Java library that provides sparse and dense matrix classes, as well as a large number of calculations for linear algebra like matrix multiplication or matrix inverse. Operations such as mean, correlation, standard deviation, replacement of missing values or the calculation of mutual information are supported, too.

UJMP Matrix

Quick Links

Documentation Download Professional Support

Description

The Universal Java Matrix Package provides various visualization methods, import and export filters for a large number of file formats, and even the possibility to link to JDBC databases. Multi-dimensional matrices as well as generic matrices with a specified object type are supported and very large matrices can be handled even when they do not fit into memory. Find out more and take a look at the feature list or compare it to other Java matrix libraries. We have also done a benchmark for matrix multiplication and matrix transposition which can be found here.

A central concept of UJMP is the separation of interfaces, abstract classes and their implementations, which makes it very easy to exchange the underlying data storage Thus, a matrix in our framework can be an array of values in main memory, a file on disk or a table in an SQL database. In fact, the actual storage implementation becomes secondary and UJMP can even integrate other matrix libraries such as Jama or Colt, making UJMP's visualization and import and export filters available to these libraries.

UJMP can decide to redirect calculations to other matrix libraries, depending on matrix size and computer hardware. UJMP uses multiple threads for calculations, which results in much better performance compared to JAMA or Colt on modern hardware.

UJMP also includes interfaces to Matlab, Octave and R, which makes it easy to perform calculations not available in Java.

Java Data Mining Package

UJMP is the mathematical back-end of the Java Data Mining Package (JDMP), which is our second open source Java project. While UJMP provides basic operations for processing matrices, JDMP concentrates on "higher-level" objects, such as Samples, DataSets and Algorithms for machine learning, e.g. clustering, classification and optimization. JDMP's central philosophy is that everything is a matrix (or can be converted into a matrix), which makes it possible to benefit a lot from UJMP's features. E.g. it becomes feasible to analyze data that does not fit into main memory, since UJMP can link to files on disk or databases. Like UJMP, also JDMP makes use of interfaces and abstract classes to allow for easy integration of other libraries like Weka or Mallet.

JDMP DataSet

Screenshots

UJMP Mandelbrot Matrix

Mandelbrot Matrix

Visualization of the Mandelbrot set.

UJMP Random Matrix

Random Matrix

This matrix is filled with random values.

UJMP Image Matrix

Image Visualization

Matrix imported from a JPG image.

UJMP Big Sparse Matrix

Big Sparse Matrix

Quadratic matrix with 100,000 rows and columns. Only some values are different from 0.

UJMP Cosine Similarity Matrix

Cosine Similarity Matrix

100 random vectors of length 10 have been created with a correlation factor of 0.1. The image shows the pairwise cosine similarity between these vectors.

UJMP Graph Adjacency Matrix

Adjacency Matrix

This graph is created from the relationship of matrix interfaces in UJMP. For example, DenseDoubleMatrix extends DenseMatrix and DoubleMatrix.

Feature List

To find out if UJMP is the right choice for you, take a look at the feature list:

  • Matrix classes for different data types
    BigDecimalMatrix, BigIntegerMatrix, BooleanMatrix, ByteMatrix, DoubleMatrix, FloatMatrix, IntMatrix, LongMatrix, ObjectMatrix, ShortMatrix and StringMatrix
  • Generic Matrix class
    Use GenericMatrix<T> to specify the object type that is stored.
  • n-dimensional matrices
    You can create matrices with any number of dimensions with
    Matrix.Factory.sparse(rows, columns, size3, size4, ...)
  • Storage of more data than fits into memory
    Matrices can be mapped to hard disk or databases.
  • Store up to 263-1 elements along one axis in a matrix
    UJMP uses long values for indices instead of int.
  • Link to JDBC data sources
    Work directly with your database:
    Matrix.Factory.linkToJDBC(host, port, db, table, user, password)
  • Store meta-data for all matrices
    It is easy to label a matrix or give names to the rows and columns.
  • Interact with other matrix libraries
    Use Colt or Jama inside UJMP. In fact, you can tell UJMP to prefer a specific matrix implementation over its own classes.
  • Interfaces to other mathematical software
    Execute complex computations in Matlab, Octave, R or GnuPlot. You can use these software packages as computing or visualization engines in UJMP.
  • Import and export filters for many different file types
    TXT, CSV, HTML, XLS, XLSX, PDF, TEX (Latex), MAT (Matlab), MDB (MS Access), and many more.
  • Link to huge CSV files without loading them into memory
    This is useful when your CSV file will not fit into your machine´s memory. Use CSVMatrix or
    Matrix.Factory.linkToFile(Format.CSV, hugeCSVFile)
  • Additional collection classes
    e.g. CachedMap, RingBufferList or SoftHashMap which can be very helpful.
  • SerializedObjectMap to store more Objects than fit into memory
    This Map stores all Objects on disk, but you can use it like a normal HashMap.
  • Mapping of collection classes
    Java collection classes like Map, List or Set can be used as UJMP matrices, which makes it easy to perform calculations. E.g. you can calculate sum, average or standard deviation of a list of values in an ArrayList. Use this to convert it to a UJMP matrix:
    Matrix.Factory.linkToCollection(myArrayList)
  • Visualization
    Heatmap, line plot, scatter plot, histogram and many more. Just use myMatrix.showGUI() to bring it on the screen. Take a look at the screenshots for examples.
  • Licensed under LGPL
    you can use it in commercial applications.

Download

Depending on your needs, you can either download jar packages of the latest release of UJMP, or check out the latest source code via SVN, if you want to stay on the edge.

Download JAR Packages

The easiest way to get started with UJMP is to download ujmp-complete.jar from SourceForge.

This package contains ujmp-core, the main part of the Universal Java Matrix Package, as well as additional features such as visualization and interfaces to other libraries, SQL databases, and a lot more. If you are new to this library, you should invoke the main() method in org.ujmp.core.UJMP to get a glimpse on UJMP:

java -jar ujmp-complete.jar

Then click on “Tools – UJMP Plugins” in the menu bar and see what third party libraries are supported. Chose the tool you want, add the necessary dependencies to the class path and restart UJMP.

Download SVN Source Code

The source code of UJMP is available via anonymous SVN from Sourceforge, and you can view the SVN repository in your web browser, if you like.

As you can see, there are many sub-projects in the repository, but probably you don’t need all of them.
You should check out ujmp-core and ujmp-gui first, the other projects are needed only, if you want to use other libraries together with UJMP.
E.g. if you want to use JAMA, you would have to check out the ujmp-jama project.

If you use Maven for software development, you should also check out the ujmp project, which provides the parent pom files for all sub-projects. All projects provide a .project file for Eclipse, which makes it easy to download them into this IDE.

Here is the command line instruction set to get the basic version of UJMP:

svn co https://svn.code.sf.net/p/ujmp/code/trunk/ujmp-core ujmp-core
svn co https://svn.code.sf.net/p/ujmp/code/trunk/ujmp-gui ujmp-gui

If you use Maven, check out this project, too:

svn co https://svn.code.sf.net/p/ujmp/code/trunk/ujmp ujmp

Examples

Take a look at the following examples, this is the fastest way to learn about the Universal Java Matrix Package. They are taken from ujmp-examples freshly from SVN:

ExploreLocalhostExample.java

// create a matrix representing the local computer with its services
Matrix localhost = Matrix.Factory.localhostMatrix();

// show on screen
localhost.showGUI();

// double click on matrix entries to navigate and see how everything is
// a matrix. Have fun exploring, but be careful when you change data!

BigSparseMatrixExample.java

// create a very large sparse matrix
SparseMatrix m = SparseMatrix.Factory.zeros(100000000, 100000000);

// set some values to 1
for (int i = 0; i < 10000; i++) {
	m.setAsDouble(1, MathUtil.nextInteger(0, 100000), MathUtil.nextInteger(0, 100000));
}

// show on screen
m.showGUI();

ExtractExcelDataExample.java

// find all Excel files in one directory
File[] files = new File("c:/temp/").listFiles();

// create matrix to store result
Matrix result = Matrix.Factory.zeros(files.length, 2);

// iterate over all files
for (int i = 0; i < files.length; i++) {

	// import file as matrix
	Matrix m = Matrix.Factory.importFromFile(files[i]);

	// store file name in result matrix
	result.setAsString(files[i].getName(), i, 0);

	// search for "Invoice"
	if (m.containsString("Invoice"))

		// extract value at row 10 and column 3 and store in result
		result.setAsDouble(m.getAsDouble(10, 3), i, 1);
}

// display result on screen
result.showGUI();

ImageMatrixExample.java

// get an image
InputStream is = ClassLoader.getSystemClassLoader().getResourceAsStream("org/ujmp/examples/data/bigdata.jpg");

// load image into matrix. of course, this works with files, too.
Matrix imageMatrix = new ImageMatrix(is);

// show on screen
imageMatrix.showGUI();

is.close();

MandelbrotMatrixExample.java

// create a matrix from the Mandelbrot set
Matrix m = new MandelbrotMatrix();

// show on screen
m.showGUI();

TreeMatrixExample.java

// create a TreeMatrix with Strings as elements
TreeMatrix<String> treeMatrix = new DefaultTreeMatrix<String>();

// create data
treeMatrix.setRoot("root");
treeMatrix.addChild("root", "child1");
treeMatrix.addChild("root", "child2");
treeMatrix.addChild("root", "child3");
treeMatrix.addChild("child1", "subChild11");
treeMatrix.addChild("child1", "subChild12");
treeMatrix.addChild("child1", "subChild13");
treeMatrix.addChild("child2", "subChild21");
treeMatrix.addChild("child3", "subChild31");
treeMatrix.addChild("child3", "subChild32");
treeMatrix.addChild("subChild12", "subSubChild121");
treeMatrix.addChild("subChild12", "subSubChild122");
treeMatrix.addChild("subSubChild122", "subSubSubChild1221");

// show on screen
treeMatrix.showGUI();

RandomMatrixExample.java

// create matrix with random values between 0 and 1
Matrix rand = Matrix.Factory.rand(100, 10);

// create matrix with random values between -1 and - 1
Matrix randn = Matrix.Factory.randn(100, 10);

// show on screen
rand.showGUI();
randn.showGUI();

CosineSimilarityMatrixExample.java

// create matrix with 10 correlated columns, 100 rows, correlation 0.1
Matrix correlated = Matrix.Factory.correlatedColumns(100, 10, 0.1);

// calculate similarity and store in new matrix,
// ignore missing values if present
Matrix similarity = correlated.cosineSimilarity(Ret.NEW, true);

// show on screen
correlated.showGUI();
similarity.showGUI();

GraphMatrixExample.java

// create a GraphMatrix with Strings as nodes and Doubles as edges
GraphMatrix<String, Double> graphMatrix = new DefaultGraphMatrix<String, Double>();
graphMatrix.setLabel("Interface Inheritance Graph");

// collect all matrix interfaces from UJMP
Class<?>[] classArray = new Class[] { DenseMatrix.class, DenseMatrix2D.class, Matrix.class, Matrix2D.class,
		SparseMatrix.class, SparseMatrix2D.class, BaseBigDecimalMatrix.class, BigDecimalMatrix2D.class,
		DenseBigDecimalMatrix.class, DenseBigDecimalMatrix2D.class, SparseBigDecimalMatrix.class,
		SparseBigDecimalMatrix2D.class, BigIntegerMatrix.class, BigIntegerMatrix2D.class,
		DenseBigIntegerMatrix.class, DenseBigIntegerMatrix2D.class, SparseBigIntegerMatrix.class,
		SparseBigIntegerMatrix2D.class, BooleanMatrix.class, BooleanMatrix2D.class, DenseBooleanMatrix.class,
		DenseBooleanMatrix2D.class, SparseBooleanMatrix.class, SparseBooleanMatrix2D.class,
		ByteArrayMatrix.class, ByteArrayMatrix2D.class, DenseByteArrayMatrix.class,
		DenseByteArrayMatrix2D.class, SparseByteArrayMatrix.class, SparseByteArrayMatrix2D.class,
		ByteMatrix.class, ByteMatrix2D.class, DenseByteMatrix.class, DenseByteMatrix2D.class,
		SparseByteMatrix.class, SparseByteMatrix2D.class, CharMatrix.class, CharMatrix2D.class,
		DenseCharMatrix.class, DenseCharMatrix2D.class, SparseCharMatrix.class, SparseCharMatrix2D.class,
		DoubleMatrix.class, DoubleMatrix2D.class, DenseDoubleMatrix.class, DenseDoubleMatrix2D.class,
		SparseDoubleMatrix.class, SparseDoubleMatrix2D.class, FloatMatrix.class, FloatMatrix2D.class,
		DenseFloatMatrix.class, DenseFloatMatrix2D.class, SparseFloatMatrix.class, SparseFloatMatrix2D.class,
		GenericMatrix.class, GenericMatrix2D.class, DenseGenericMatrix.class, DenseGenericMatrix2D.class,
		SparseGenericMatrix.class, SparseGenericMatrix2D.class, IntMatrix.class, IntMatrix2D.class,
		DenseIntMatrix.class, DenseIntMatrix2D.class, SparseIntMatrix.class, SparseIntMatrix2D.class,
		LongMatrix.class, LongMatrix2D.class, DenseLongMatrix.class, DenseLongMatrix2D.class,
		SparseLongMatrix.class, SparseLongMatrix2D.class, ObjectMatrix.class, ObjectMatrix2D.class,
		DenseObjectMatrix.class, DenseObjectMatrix2D.class, SparseObjectMatrix.class,
		SparseObjectMatrix2D.class, ShortMatrix.class, ShortMatrix2D.class, DenseShortMatrix.class,
		DenseShortMatrix2D.class, SparseShortMatrix.class, SparseShortMatrix2D.class, StringMatrix.class,
		StringMatrix2D.class, DenseStringMatrix.class, DenseStringMatrix2D.class, SparseStringMatrix.class,
		SparseStringMatrix2D.class };

// find out how interfaces extend one another
for (Class<?> c1 : classArray) {
	for (Class<?> c2 : classArray) {
		if (c2.getSuperclass() == c1) {
			// add edge when class2 extends class1
			graphMatrix.setEdge(1.0, c1.getSimpleName(), c2.getSimpleName());
		}
		for (Class<?> c3 : c2.getInterfaces()) {
			if (c1 == c3) {
				// add edge when class2 implements class1
				graphMatrix.setEdge(1.0, c1.getSimpleName(), c2.getSimpleName());
			}
		}
	}
}

// show on screen
graphMatrix.showGUI();

What It's Not

  • Not finished
    While some parts of UJMP are already pretty stable, a lot of development is still going on in other parts. There is no guarantee that everything will be working as expected.
  • Not well documented
    Since many things are going to change anyway, I didn't bother much to write comments for functions or JavaDoc for parameters.
  • Not a lightweight matrix library
    If you just need linear algebra for double matrices, there are probably better choices, such as EJML or Ojalgo. UJMP is rather a swiss army knife for data processing. It will help you when you want to find all Excel files which contain the text "Invoice", extract the total amount, and show a nice overview. UJMP can do this with 8 lines of code.

Alternatives

There are many matrix packages available for Java. If you are interested in alternative matrix packages, this link may be helpful: Overview of Matrix Libraries for Java

Contributors

Holger Arndt

Project manager, UJMP core package, UJMP GUI package, interfaces to other libraries
Holger's Homepage

Markus Bundschus

Strategical consultancy, marketing
Markus's Homepage

Andreas Nägele

Core package, GUI package, test cases

Rand Huso

Method for generalized matrix inversion
Rand's Homepage

Frode Carlsen

Block matrix implementation and multiplication
Frode's LinkedIn Profile

Support UJMP

You can use UJMP free of charge. However, if you like it, we would appreciate to receive a small donation from you. Please support open source software. Thank you very much!

Big Data AnalyticsData MiningMachine LearningArtificial IntelligenceNeural Networks