The human way of resolving difficult situations is to compromise. That is not God's way: He never compromises on truth. However, divine truth often presents us with two issues which are different but both true and need to be held in tension. For example: God loves sinners, but hates sin; God is invisible but God the Son is the image of the invisible God (Colossians 1:15) ... etc.
 
When the Pharisees tried to trap Jesus over the contentious issue of the Roman poll tax, Jesus refused to say that the Jews should not pay (which would have raised a charge of treason against Him), or that they should (which could have sparked a riot of the Jews against Him ... and then Roman action against Him for 'disturbing the peace').
 
He called for a denarius, the coin which would be used for paying the Roman tax. (There was another currency, the shekel, in Israel at that time but it was used exclusively in the Temple Tax, and it did not have any Roman inscription on it). But Caesar's head was on the common denarius coin. Jesus resolved the question of whether they should pay the civil tax by telling them to give back to Caesar what belonged to Him and likewise for whatever belonged to God. They were amazed! Jesus had not resolved the tension which the people felt, living under an occupying army, but He had explained it, had not compromised any truth and gave them two action points which were both valid.
 
So we must also pay taxes because, although the authorities may be against people who believe in Jesus, the authorities are appointed by God (Romans 13:6-7) for our good. God does not expect us to give everything to the state, but neither should we give nothing ... we pay what they ask according to the law. At the same time we should also give generously to God out of gratitude to Him (2 Corinthians 8:1-5). It honours God to pay what we owe to the world as well as giving sacrificially to Him. Withholding money from God is stealing, as much as failing to pay taxes (Malachi 3:8-10).