Text Summarizer with Hugging Face Transformers
This guide demonstrates how to build a text summarization tool using the Hugging Face transformers library. It utilizes pre-trained Deep Learning models (like BART or T5) to generate concise summaries of long text documents.
Modules Used:
transformers: Provides general-purpose architectures for Natural Language Understanding (NLU).torch: The PyTorch deep learning framework (required backend).- argparse: To handle command-line arguments.
Installation
You need to install transformers and a deep learning backend (PyTorch is recommended).
pip install transformers torchThe Code
Save this as summarizer.py.
from transformers import pipeline
import argparse
import os
import sys
# Suppress warnings from transformers (optional)
import logging
logging.getLogger("transformers").setLevel(logging.ERROR)
def summarize_text(text, max_length=130, min_length=30):
print("Loading model (this may take a moment first time)...")
try:
# Initialize the summarization pipeline
# By default, this uses 'sshleifer/distilbart-cnn-12-6'
summarizer = pipeline("summarization")
# Check if text is too short
if len(text.split()) < min_length:
print("Warning: Text is shorter than the minimum summary length.")
# Generate summary
# do_sample=False uses beam search (deterministic)
summary = summarizer(text, max_length=max_length, min_length=min_length, do_sample=False)
return summary[0]['summary_text']
except Exception as e:
return f"Error during summarization: {e}"
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Text Summarizer using Hugging Face Transformers")
parser.add_argument("input", help="Text string OR path to a text file")
parser.add_argument("--max", type=int, default=130, help="Maximum length of summary (default: 130)")
parser.add_argument("--min", type=int, default=30, help="Minimum length of summary (default: 30)")
args = parser.parse_args()
# Determine if input is a file or raw text
content = args.input
if os.path.exists(args.input) and os.path.isfile(args.input):
try:
with open(args.input, 'r', encoding='utf-8') as f:
content = f.read()
print(f"Loaded {len(content)} characters from '{args.input}'")
except Exception as e:
print(f"Error reading file: {e}")
sys.exit(1)
if not content.strip():
print("Error: Input text is empty.")
sys.exit(1)
print("\n--- Summary ---")
result = summarize_text(content, args.max, args.min)
print(result)
print("---------------")Usage
# Summarize a short string
python summarizer.py "The quick brown fox jumps over the lazy dog. This is a story about a fox and a dog."
# Summarize a text file
python summarizer.py article.txt --max 100 --min 50