Accurate early season yield prediction is important for farm resource management (e.g. nitrogen and water management) and market planning. Currently, crop yields are forecast based on field surveys, which is labor intensive and time consuming. Satellite remote sensing, on the other hand, provides consistent, spatially extensive measurements covering the visible and infrared spectrum, and thus has great potential for crop yield analysis. My goal is to build advanced machine learning models, such as deep learning, to extract the informative features (e.g. tree vigor and flower condition) from the time-series remote sensing imagery, and then combine them with other input data (e.g. weather data, historical yields) for the end-of-season yield prediction.