Sandia National Laboratories conducted the first-ever blind comparison of seven commercial PV modeling software, showing that differences in weather treatment, system modeling, reductions and assumptions increase as system complexity increases. The research highlights that the choice of software should take into account the complexity of the project, workflow and modeling features, rather than relying solely on rankings.
A group of scientists from the U.S. Department of Energy’s Sandia National Laboratories conducted a comprehensive assessment of seven PV modeling software tools – 3E SynaptiQ, PlantPredict, PVsyst, RatedPower, SAM, SolarFarmer and Solargis Evaluate – and found that their performance varies as system complexity increases.”
“This is the first-ever blind and independent comparison of commercially used PV software, with predictions submitted directly by the software vendors,” said the study’s corresponding author, Marios Theristis, pv magazine. “We didn’t rank the tools, but we focused on how different modeling features and assumptions affect predictions.”
“We compiled summary tables of software features and then compared the providers’ predictions,” he continued. “We found that the results for simple systems are closely aligned, by which we mean small-scale, monofacial, fixed-tilt, flat-terrain, and small-scale systems, while the differences increase as systems become more complex and linked to specific modeling choices and software features.”
In the study “Feature overview of photovoltaic modeling software using blind performance assessment”, published in Solar energythe Sandia group explained that, unlike previous studies that focused on small systems, single sites, or anonymous participants, their work provides a transparent feature-level comparison of commonly used PV modeling software that supports both pre-construction and post-construction activities.
The scientists categorized software functions into weather and radiation, DC system modeling, AC system modeling and derates. The software tools were tested using one year of data from two monofacial, south-facing, fixed-tilt systems in Albuquerque, United States and an undisclosed location in Germany, with capacities of 15.4 kW and 14.5 MW, respectively.
The measurements were performed independently, with instrument details withheld from the software providers, allowing an unbiased blind comparison of PV modeling performance on different software platforms, the research team said. Weather and radiation data were also filtered before being distributed to software vendors.
The analysis showed that PV modeling results vary significantly between software due to differences in weather treatment, system modeling, inverter assumptions and user-specified reductions, while highlighting the critical influence of both software design and user choices on predicted energy results. Additionally, the blind modeling comparison revealed differences between software in plane of array (POA) irradiation transposition, module temperature, DC/AC power, and reductions.
For example, weather modeling varied due to different libraries, transposition models, and assumptions about air mass, albedo, and location, with median POA residuals ranging from 14.65 to 6.06. The DC system characteristics were generally consistent, but the shading and temperature models varied. In contrast, the AC system modeling differed in terms of inverter efficiency, handling of clipping, and curtailment adjustments. Furthermore, the shading approximations were found to vary through spanning, irradiance decomposition, and terrain assumptions, introducing uncertainty.
“Our findings underscore the need for continuous, independent, and rigorous validation of modeling methods, comparing software tools to complex, real-world systems,” said Theristis. “Ultimately, the ‘right’ tool depends on the complexity of the project, the workflow and the surrounding software ecosystem.”
This content is copyrighted and may not be reused. If you would like to collaborate with us and reuse some of our content, please contact: editors@pv-magazine.com.
