I really don't think Lincoln said that (assuming he did, Lincoln certainly didn't try to apply that to the question of slavery, so under what context did he say it?). My sense of history smell is calling shenanigans and a quick search shows it's one of those things that was probably made up as I couldn't easily find an actual source.
Sorry to be slightly off-topic here, but it grinds my gears when using a made up quote by someone famous who didn't say actually it (and never likely did historically) doesn't add any real value to a conversation.
Indeed, that's why I was uncertain -- I've definitely seen it attributed to Lincoln, but that alone is not enough to convince me that he was actually the originator of the observation.