My money's on this connecting to Hell somehow

"Apocryphon" has religious overtones, and while "caustic" usually refers to chemical burns or corrosive substances, I believe it originally meant "combustible" or "burning". So religion + fire = Hell?

The Advent Calendar text saying it "causes ink to run, and words to shift and change" makes me think of the Iron Republic, although I suppose they're more about new laws changing the way things are than about altering records of how things were.

One of the Marigold Knights was a historian exiled for trying to "create a comprehensive analysis of the revolution", contrary to Hell which "rewrites its doctrine regularly". It's not much of a leap from there to rewriting history ...