Housing starts are up 24 percent from a year ago, but residential construction employment is up only 3.1 percent, according to the U.S. Commerce and Labor Departments. Former construction workers headed to higher paying jobs, such as the trucking industry. Some in the Southwest headed north to the newly thriving energy sector, said Marquis Jackson. Others headed to highway construction jobs in Texas.

(Read More: Job Creation Surges as Rate Falls to 7.7%)

In Las Vegas, where Pardee Homes is building 150 percent more homes this year than they did last year, finding workers is increasingly difficult.

“We lost quite a bit of labor to the oil fields and to places like Wyoming and North Dakota, where you would not expect it to go,” noted Klif Andrews, Pardee’s Las Vegas president.

Andrews said he is paying workers five to ten percent more now, and that has pushed his home prices higher. “We’ve raised median home prices up over 15 percent, so we’ve been able to stay a little bit ahead of it, but cost increases, it’s not just labor, it’s also materials,” he added.

Nationwide, the median price for newly built homes rose 2 percent in January and new home prices now far exceed the median price for existing homes.

(Read More: Housing: Picking Up the Pieces)

Another factor is weighing on workers. Contractors, who hire the specialty workers for builders, are small businessmen themselves. In order to fund their operations, they need to hire people to have ready, and that takes capital. Unfortunately, despite the housing recovery, banks are still not lending freely to builders. Builders are trying to find ways around that, by paying subcontractors more frequently, essentially bankrolling them so they can hire more workers.

“The wheels are starting to work, but all the gears are not working in alignment,” said Marquis Jackson.

Article source: http://www.cnbc.com/id/100537494