URL summariser

URL Summariser (or "summarizer" if you're across the pond) is a very simple microservice for summarising webpages. All code available on GitHub. Many similar tools exist, but this one is has some specific features so that it can be easily used in categor.ai.

Usage

GET https://summarise.amar.io/https://example.com

Replace https://example.com with any URL to summarise. The request is blocking.

Responses:

• 200; [raw text summary]
• 400; No URL provided
• 400; URL must be absolute
• 400; Could not fetch webpage

Features

• YouTube links -- the video transcript is fetched (English, if more than one language) and summarised instead
• HackerNews links -- the page that is linked to is summarised instead
• Redirects are followed
• Text is extracted using Mozilla's Readability.js
• Summaries are around 100 words by default
• If the text to summarise is above the OpenAI token limits, the text is recursively chopped up and we summarise summaries
• Please don't summaries books with this, or it'll burn through my credits. If there are more than 10,000 words (approx 13k tokens in English), it'll only consider the first 1,000 words. Feel free to remove this safeguard if you self-host.
• If OpenAI is at capacity, we automatically retry the failed queries
• If we hit OpenAI rate limits, we don't send back an error, rather we retry with 5 second gaps so your request will just take longer

Please feel free to use this for free but please be reasonable. Caveats:

• I make no guarantees about uptime or results
• If my OpenAI billing caps are hit, I may turn this off
• I host this on one of the small free instances on fly.io, which has its own limits. If those are hit, please feel free to self-host, or launch your own on fly.io (config in repo)

TODO

• Reddit comments vs links
• Handle HackerNews posts with no link
• Optionally summarise HackerNews comments instead of the link
• Add safeguards against links with loads of text (e.g. books, which immediately hits rate limits)

Apps Script for Google Sheets

If you have a spreadsheet full of links, and you want to have a column be the summaries for those, you can use this script and use the formula =summariseURL(A1) where A1 is a cell with a URL.

function summariseURL(url) {
  if (!url)
    return "";

  const cache = CacheService.getScriptCache();
  const cachedResponse = cache.get(url);

  if (cachedResponse !== null)
    return cachedResponse;

  try {
    const response = UrlFetchApp.fetch('https://summarise.amar.io/' + url);
    const contentText = response.getContentText();
    cache.put(url, contentText);
    return contentText;
  } catch (error) {
    return error.toString();
  }
}

Amar Memoranda > URL summariser

Yousef Amar