A major shortfall of vision-based inspection solutions is the lack of scale information, required to resolve inspection regions to a physical scale. To address this challenge, a learning-based scale estimation technique is proposed. The underlying assumption is that the surface texture of structures, captured in images, contains enough information to estimate scale for each corresponding image (e.g., pixel/mm). This permits the training of a regression model to establish the relationship between surface textures, captured in images, and scales. A convolutional neural network is trained to extract scale-related features from textures captured in images. Then, the trained model can be exploited to estimate scales for all images that are captured from a structure's surfaces with similar textures. The capability of the proposed technique is demonstrated using data collected from surface textures of three different structures. An average scale estimation error, from images of each structure, is less than 15%.