From Fallen London Wiki

the affairs of the box carrousel and card-drawing are a decent place to increase favors:criminals.

It took me about a week and a half to go from ~20 to 40