Yield and its prediction is one of the most important tasks in grapevinebreeding purposes and vineyard management. Commonly, this trait is estimatedmanually right before harvest by extrapolation, which mostly islabor-intensive, destructive and inaccurate. In the present study an automatedimage-based workflow was developed quantifying inflorescences and singleflowers in unprepared field images of grapevines, i.e. no artificial backgroundor light was applied. It is a novel approach for non-invasive, inexpensive andobjective phenotyping with high-throughput. First, image regions depicting inflorescences were identified and localized.This was done by segmenting the images into the classes "inflorescence" and"non-inflorescence" using a Fully Convolutional Network (FCN). Efficient imagesegmentation hereby is the most challenging step regarding the small geometryand dense distribution of flowers (several hundred flowers per inflorescence),similar color of all plant organs in the fore- and background as well as thecircumstance that only approximately 5% of an image show inflorescences. Thetrained FCN achieved a mean Intersection Over Union (IOU) of 87.6% on the testdata set. Finally, individual flowers were extracted from the"inflorescence"-areas using Circular Hough Transform. The flower extractionachieved a recall of 80.3% and a precision of 70.7% using the segmentationderived by the trained FCN model. Summarized, the presented approach is a promising strategy in order topredict yield potential automatically in the earliest stage of grapevinedevelopment which is applicable for objective monitoring and evaluations ofbreeding material, genetic repositories or commercial vineyards.