The biggest challenge most buyers face when purchasing a home? Coming up with that big down payment. Even if your mortgage lender only requires a down payment of 5 percent, that still comes out to $10,000 for a modestly priced home of $200,000. Many buyers simply don't have that much cash lying around.

If you have a 401(k) plan at work, though, you might have a convenient source for down payment funds. You are allowed to borrow money from this retirement account for a down payment. You just have to pay back your loan -- with interest -- on time to avoid any penalties or taxes.

But does doing this make financial sense? That depends upon how badly you want the home, how close you are to retirement and how certain you are that you can pay back the loan on time.

Heather McRae, senior loan officer with Chicago Financial Services in Chicago, said that a 401(k) loan has helped several of her clients gather the funds they need for down payments. She considers it a smart financial move for borrowers who know they can handle the payback schedule.

"If you don't have the money for a down payment and you don't have family members who are kind enough to gift you the down payment, you're kind of out of luck," McRae said. "The 401(k) loan is often the best option for these buyers. If you haven't saved the money for a down payment and you've fallen in love with a property, the 401(k) can make the purchase work."

How it works

You can take out a loan from your 401(k) account for up to $50,000 or half of the value of your account, whichever figure is less. You will have to pay interest on the money you borrow, but you won't have to pay any taxes or penalties on this amount, as long as you pay the money back on time. And that interest you pay? It goes back into your 401(k) account.

How long you'll have to pay back the money depends on your plan. Some plans might give you five years to pay back your loan, which you'll do through regular monthly payments, the same as with a mortgage or auto loan.

There can be complications, though. If you have to leave your place of employment sooner than expected, you might have to pay back the remainder of your loan in just 60 to 90 days. If you can't do this, your loan will then be considered an early withdrawal, which means that you'll have to pay both taxes and penalties on whatever amount you still owe.

But if you can afford the payments and you’re far from retirement age, tapping your 401(k) could be a solid option for down payment funds.

Taking advantage of low interest rates

Eric Meermann, portfolio manager with the Scarsdale, New York, office of Palisades Hudson Financial Group, says that because housing prices across the country remain at reasonable levels and mortgage rates remain at historic lows, this is a good time to buy a home.

If all that's preventing buyers from taking advantage of this market is a lack of down payment funds, taking a loan from a 401(k) plan can be a smart financial move, Meermann said. Buyers who wait might find that both fixed mortgage rates and adjustable mortgage rates have risen by the time they’re ready to buy.

"It can make sense to get you into the house now at a presumably lower price and lock in a low interest rate," Meermann said. "This is opposed to taking the additional few years it may take to get the money together for the down payment otherwise, risking higher home prices and higher interest rates."

The traditional hurdle

Ed Hoffman, president of Wholesale Capital Corporation in Moreno Valley, California, said that down payments have historically been the biggest hurdle that first-time home buyers face. That’s because these buyers can’t rely on the sale of a previous home to cobble together the dollars they need for that down payment.

Some first-time buyers rely on financial gifts from their parents or other family members to cover their down payment costs, while others use their tax returns to help boost these dollars. But for those who can’t rely on either of these, borrowing from a 401(k) loan is often the best option, Hoffman said.

“It’s not easy for young people to save that money for a down payment,” Hoffman said. “Even if you’re making $50,000 a year, with buying a car, renting an apartment and paying for the other necessities of life, it’s not easy to come up with the down payment money. By the time you do save up that $7,000 or so, the price of the homes you want might rise by $25,000.”

The drawbacks

Not everyone agrees that borrowing from a 401(k) even for a down payment makes sense.

Matthew Carbray, managing partner with Ridgeline Financial Partners in Avon, Connecticut, said that when buyers remove a big chunk of money from their 401(k) plans they reduce the power of the compounding that helps their retirement accounts grow faster.

Say buyers borrow the maximum $50,000 from their retirement account for a down payment. That money is no longer compounding, meaning that it is no longer earning interest. The impact of that can be big when retirement nears and those buyers who spent five years repaying a 401(k) loan don’t have quite as much saved in that account as they had hoped, Carbray said.

"It should not be utilized for a home purchase because it is called a retirement account and not a general purpose account," Carbray said.

Gregory Ostrowski, managing partner with Scarborough Capital Management in Annapolis, Maryland, said that borrowing against a 401(k) account could cost buyers more than they think.

He gives this example: A 40-year-old with $60,000 in her 401(k) account borrows $15,000 from it at 6 percent interest for five years. To make the loan payments, she reduces her monthly 401(k) contribution from $750 to $460 and continues to receive her employer match of 5 percent.

Ostrowski says that this loan could cost the borrower $85,883 by the time she retires. That's how much more this borrower could have made if she had continuously invested in her account at an average rate of return of 7 percent, Ostrowski said.

Still … it’s not the worst plan

Kyle Winkfield, managing partner with the Rockville, Maryland, office of financial planning firm of O’Dell, Winkfield, Roseman and Shipp, said that he recommends that his clients first pursue other options before borrowing against their 401(k) plans, with the best being to plan ahead before buying a home.

“Maybe you know a couple of years out that you’re going to be ready to buy a home, now you can start saving for a down payment,” Winkfield said. “You can start paying down debt and start saving money in an account that you can use for your down payment when you’re ready to move in two to three years.”

That’s the ideal. But it’s not always realistic, Winkfield said. Many of his clients walk into a model home, fall in love and want to buy that home, only to find that they don’t have nearly enough money in their savings account for a down payment.

These same clients then see that big chunk of money in a 401(k) loan, money that they know they can tap for those elusive down payment dollars.

“I tell them that borrowing against a 401(k) plan isn’t as good as some other options,” Winkfield said. “But it’s not the end of the world, either, as long as you pay back what you borrow on time. We have to be realistic: Most people are not going to have the same amount of money saved up in a traditional savings account as they will in their 401(k) accounts. The 401(k) is forced saving, and that is usually the biggest chunk of money that people have.”