These cannot be found where the city still stands.
From Fallen London Wiki
Does Fallen London not have shattered china and broken gas-lamps already? You could very easily break a few of your own teacups and voilĂ ! Relics of the Fifth City!
I just find it funny that the shopkeeper is bewildered and wants so badly scraps you can find by dumpster diving.