As many as one in five primary schools needs to be rebuilt or replaced.
A report for Scottish council leaders found that 400 of the country's 2,000 primaries needs to be rebuilt.
Ministers are expected to identify around 20 primary schools across Scotland that they have prioritised for urgent work.
But that is just five per cent of the number of schools the report says should be replaced.
The report, obtained by Scotland on Sunday, was discussed at a Convention of Scottish Local Authorities (Cosla) leaders' meeting in Edinburgh last week.
Education secretary Fiona Hyslop has already been accused of failing to rise to the challenge of delivering new school buildings. And the SNP is said to have struggled to fulfil its manifesto promise to pay for new schools through a new funding mechanism, the Scottish Futures Trust (SFT).
Labour education spokesman Des McNulty described the situation as "a scandalous neglect of children".
A spokesman for Ms Hyslop said Scotland was dealing with decades of lack of investment and claimed the government had lifted 100,000 pupils out of unsatisfactory school accommodation.
The spokesman added that three-quarters of schools were now rated as "satisfactory or better".
But a Labour spokesman added: "The SNP are failing Scotland's pupils, parents and teachers with their failure on school building, yet they continue to try and mislead voters about their school building programme.
"The fact is that school building has stalled under the SNP due to the disastrous Scottish Futures Trust.
"This has meant thousands of construction jobs lost as well as pupils facing a future in schools that need to be replaced."
Conservative MSP Liz Smith, shadow minister for children, schools and skills, said: "Too many schools are past their sell-by date and the scale of the primary school problem is particularly worrying.
"I fully appreciate that there are budget constraints but we need an assurance from the SNP that the criteria for deciding the priorities are both clear and transparent."
