Deciding whether to refinance your mortgage means considering your personal situation, the prevailing interest rate environment — and something that really hits close to home: closing costs.

What are refinance closing costs?

You may pay as much as 2%-6% of your outstanding principal in mortgage refinance fees, known as closing costs, though the total can vary by state and lender. It's not a massive single charge, but a pile of small costs that quickly add up.

If you decide to lock in a new, lower mortgage rate, switch loan products or tap equity by refinancing, here are the closing costs to watch out for.

» MORE: How to refinance your mortgage

Early repayment fees

If refinancing will result in an early repayment fee from your mortgage lender, you may want to reconsider. Prepayment penalties aren't very common and are usually applicable only within the first three to five years of a mortgage.

FHA and VA loans, and other mortgages backed by a federal agency, generally cannot include a prepayment penalty. And in some states, they are illegal. But it’s one thing you want to be aware of before beginning the refinance application process.

» MORE: Mortgage closing costs, explained

Discount points

Points are interest payments based on the total amount of your mortgage. You can prepay points to lower your mortgage loan’s long-term interest rate — these are called discount points. In other words, you pay now instead of later.

If you plan on staying in a home for a long time, paying for discount points to lower your mortgage rate may work to your advantage. Remember, when you buy points, it alters the break-even period for your refinance, so be sure to do the math.

» MORE: Mortgage calculator: Should I buy points?

Lenders can also charge points on a mortgage refinance simply to make more profit — without reducing the mortgage’s interest rate. If you see points on your Loan Estimate and didn't buy discount points, you've got room to negotiate those fees.

Origination fees

The mounds of paperwork you’ll face when closing on your mortgage refinance come at a price. Lenders often charge origination fees to cover the cost of processing your loan and obtaining a credit report.

These origination fees, sometimes referred to as an administration fee, application fee, underwriting fee or document preparation fee, can increase your closing costs even further. The good news is that all origination fees are open to negotiation, so don't be afraid to discuss them with your lender.

Appraisal and inspection fees

There is due diligence performed by the lender to ensure that your home is properly valued. You guessed it — you’ll pay for that, too.

Among the refinance closing costs, you may see a home appraisal fee, often between $300 and $500. If you bought your home recently and already have an appraisal on file, sometimes this service can be waived. However, if you suspect your home has gained value, you may want to have your home reappraised.

The mortgage lender may also require a termite and pest inspection, as well as a general review of the home’s condition by a property inspector or engineer. The charges for these services can be several hundred dollars as well.

» MORE: Fannie Mae and Freddie Mac refinances with fewer fees

Mortgage and title insurance fees

While you will already have homeowner’s insurance due to your existing mortgage, the refinance lender will likely confirm the coverage, protecting itself in the event the house is damaged or destroyed.

Mortgage loans backed by government agencies, such as the Federal Housing Administration (FHA) or the Department of Veterans Affairs (VA) require the payment of mortgage insurance — once again for the benefit of the lender.

If you have a conventional mortgage but put less than 20% down, you will be required to pay for private mortgage insurance (PMI) to protect the lender in the event of a default. The fees for lender’s insurance vary by the entity involved and ranges from 0.55% to 2.25%.

And there is one more protection policy involved when refinancing a mortgage: title insurance. Courthouse records are searched to determine if you have valid ownership of the house and land and to ascertain if there are any liens against the property.

Title insurance covers the cost of any errors made in such investigations in order to protect the mortgage lender — and optionally the owner. Costs vary by loan value, down payment, property location and provider.

You may also be required to pay for a survey of the property and improvements, unless one has been recently completed. This costs about $500.

A final tip on refinance closing costs