Can AI Read Candlestick Charts Like a Trader?

The Hypothesis: Charts as Images

Technical traders spend years learning to read candlestick patterns—doji, hammers, engulfing patterns, morning stars. These visual patterns allegedly signal future price movements. But here's an intriguing question: Can a convolutional neural network learn to "see" these patterns too?

Instead of engineering features from OHLC data (open, high, low, close), what if we just showed the model the actual chart image—the same thing a human trader sees—and let it learn directly from pixels?

💡 The Idea: Treat stock prediction as an image classification problem. Feed candlestick chart images to a CNN and let it predict: will the price go UP or DOWN?

The Dataset

I created a dataset of RGB candlestick chart images, each labeled based on whether the subsequent price movement was upward or downward. The dataset was split into training, validation, and test sets to ensure proper model evaluation.

2 Classes (Up/Down)

RGB Image Format

3 Data Splits

CNN Model Type

            
candlestick_dataset/

├── train/

│   ├── up/      # Upward movement images

│   └── down/    # Downward movement images

├── validation/

│   ├── up/

│   └── down/

└── test/

    ├── up/

    └── down/

Model Architecture Evolution

I experimented with multiple CNN architectures to find the best balance between model complexity and generalization ability. Here's how the journey unfolded:

Stage 1: 3-Layer CNN from Scratch

Started with a simple architecture: three convolutional layers with max pooling, followed by a flatten layer and dense layers for classification.

Input RGB Image (height × width × 3)

Conv2D + Pool 32 filters, 3×3 kernel, ReLU, MaxPool 2×2

Conv2D + Pool 64 filters, 3×3 kernel, ReLU, MaxPool 2×2

Conv2D + Pool 128 filters, 3×3 kernel, ReLU, MaxPool 2×2

Flatten → 1D vector

Dense 128 units, ReLU

Output 2 units, Softmax (Up/Down)

Result: Test accuracy of 46.4%—barely better than random guessing for a binary classification task.

Stage 2: Simplified 2-Layer CNN

Hypothesis: Maybe the model was overfitting? I reduced the architecture to just two convolutional layers to see if a simpler model would generalize better.

Result: Faster training, but similar accuracy. The problem wasn't just model complexity.

Stage 3: Learning Rate Tuning

The key breakthrough came from systematic hyperparameter tuning. I tested multiple learning rates to find the optimal training dynamics:

Learning Rate	Train Accuracy	Val Accuracy	Test Accuracy
0.01	48.2%	45.1%	44.8%
0.005	51.3%	48.7%	49.2%
0.001 ✓	55.8%	53.2%	52.1%
0.0005	52.1%	50.4%	50.8%

The optimal learning rate of 0.001 achieved the best test accuracy of 52.1%.

            
# Final model training with optimal learning rate

from tensorflow import keras

from tensorflow.keras import layers

model = keras.Sequential([

  layers.Conv2D(32, (3, 3), activation='relu', input_shape=input_shape),

  layers.MaxPooling2D((2, 2)),

  layers.Conv2D(64, (3, 3), activation='relu'),

  layers.MaxPooling2D((2, 2)),

  layers.Conv2D(128, (3, 3), activation='relu'),

  layers.MaxPooling2D((2, 2)),

  layers.Flatten(),

  layers.Dense(128, activation='relu'),

  layers.Dense(2, activation='softmax')

])

model.compile(

  optimizer=keras.optimizers.Adam(learning_rate=0.001),

  loss='categorical_crossentropy',

  metrics=['accuracy']

)

Technology Stack

Python 3.8+ TensorFlow 2.13+ Keras NumPy Matplotlib scikit-learn Jupyter Notebook

Results & Analysis

52.1% Test Accuracy

0.001 Best Learning Rate

3 Conv Layers

~2% Above Random

"The market is a device for transferring money from the impatient to the patient."
— Warren Buffett

Honest Assessment: Why 52%?

Let's be real: 52.1% accuracy on a binary classification task is only marginally better than a coin flip. But this result is actually quite informative:

⚠️ Key Insight: The modest accuracy suggests that either (a) candlestick patterns alone don't contain strong predictive signal, (b) the patterns are too noisy for a basic CNN, or (c) markets are inherently hard to predict—which efficient market hypothesis proponents would argue.

Why This Matters

Baseline Established: We now know what a simple CNN can achieve on this task—future models can be benchmarked against this.
The Hard Truth: If stock prediction were easy, everyone would be rich. A 52% edge, compounded over many trades, can still be profitable—but transaction costs and slippage eat into thin margins.
Learning Opportunity: The project demonstrates end-to-end deep learning workflow: data prep, model building, hyperparameter tuning, and evaluation.

What I Learned

Image-based finance is viable. CNNs can process financial charts, opening doors for more sophisticated approaches.
Hyperparameter tuning matters. The jump from 46% to 52% came purely from finding the right learning rate.
Simple models have limits. A vanilla CNN may not capture the temporal dynamics that drive price movements.
Markets are efficient (mostly). If a simple model could beat the market, arbitrageurs would have exploited it away.

Future Improvements

To push beyond 52%, potential next steps include:

Transfer Learning: Use pre-trained models (VGG, ResNet) fine-tuned on candlestick data
Hybrid Models: Combine CNN with LSTM to capture both visual patterns and temporal sequences
Data Augmentation: Increase dataset size with synthetic chart variations
Multi-class Prediction: Predict magnitude of movement, not just direction
Ensemble Methods: Combine multiple models for more robust predictions
Feature Fusion: Blend image features with numerical indicators (RSI, MACD, volume)

View on GitHub View in Portfolio Discuss This Project

Can a Neural Network Read Candlestick Charts Like a Trader?