Performance Issues with layer_likelihood (sparse matrix representation)

[verginer]

The layer_likelihood function in tests has a bottleneck the indexing of the transition matrix

pathpy/pathpy/classes/multi_order_model.py

Lines 473 to 480 in 595fc2d

	transition_matrix = self.transition_matrices[l]
	factor_ = paths.paths[k][p][1]
	for s in range(len(nodes)-1):
	idx_s1 = indexmaps[l][nodes[s + 1]]
	idx_s0 = indexmaps[l][nodes[s]]
	trans_mat = transition_matrix[idx_s1, idx_s0]
	likelihood += np.log(trans_mat) * factor_

Some stats changing the type of sparse matrix used:

	dense	lil	csr	csc
layer_likelihood	2.8	3.5	10	27
estimate_order	16.8	17.6	24	42
with instantiation	29.1	30	41	51

[IngoScholtes]

Kamino cloned this issue to uzhdag/pathpy