Categories
All Softwares
Sublime Text VSCode Binaries Postman TeamViewer Firebase Visual Studio Code Contentful Tribe Circle Notion Datadog NewRelic Vmix Archbee Docker Desktop Bitly GitHub BitBucket Upsource Zapier Make Medium substack Facebook Amazon S3 Maya 3DS MAX Adobe Substance Airtable Roam Research Azure DevOps Retool Powerapps Appsheet 1Password Plex Emby Netflix Apple HomeKit Algolia Lightshot Confluence Toad tableau Data Studio Salesforce SAP Calendly Google photos Bloomberg Terminal BigQuery ML Google AutoML Tables Shopify BigCommerce Google Drive Redis Memcached Windows media player WhatsApp Heroku Render Looker Quizlet Google Analytics Auth0 Trello Elasticsearch Adobe Premiere Pro Zerotier Zoom Skype Docker Polypane Google Chrome Microsoft Edge Safari Gitbook Gmail Google Vertex AI Kdb+ Amplitude Google Docs Typora Roboflow ML Kit Azure Intercom Quicken YNAB Uptime Robot Figma npm TigerGraph Amazon Neptune Fivetran Okta YouTube LastPass Mailchimp Sendinblue Adobe Acrobat Pocket Reddit Onenote Shogun DaVinci Resolve UiPath Taliscale Adobe Lightroom FullStory LogRocket RescueTime Boxcryptor LaunchDarkly ArcGIS AWS SageMaker Tailscale NordVPN WooCommerce Twitter Dropbox Nagios Zabbix Prtg Google Cloud Webflow ActiveCampaign Quickbooks .Net Maui Airplane.dev Pipedream Evernote Autodesk AutoCAD HCL Connections Google Sheets Excel Rundeck Ansible Tower Salt Twilio Pastebin Zoho Unity3D GameMaker AWS Config GCP Cloud Asset inventory AWS GuardDuty Unreal Engine (UE4) Jira YouTrack Stytch Suite CRM Greynoise Photoshop LinkTree BlackBoard Zendesk Discord Rollout.io Disqus Oracle Fusion ERP Cloud Odoo Microsoft Dynamics Alfred Sophos Firewall UniFi Security Gateway Azure AD Doodle Office Online Power BI MicroStrategy Qlik Ampache Socrata Drone CI IOS WordPress IDM FDM Ninja Download Manager McAfee Google Meet WIX cPanel LucidChart HubSpot Landbot Typeform CCleaner Ecwid Spotify Stackstrom N8N Substance Painter Onshape SketchUp Canny Miro XMind Segment GoogleForms Adobe Illustrator MultiSim Proteus Prezi Slack Microsoft Teams SumSub JAWS Wetransfer Framer Microsoft 365 Telegram Threema Signal Lokalise Crowdin Phrase WolframAlpha Dataclay Templater Bot WorkOS FrontEgg Snorkel AI ZohoCRM Voicemod Chromatic Percy POEditor Transifex Microsoft Office Selenium vBulletin Xenforo Hightouch Logseq Bundlephobia Webpack Esbuild Rollup Session Berty WHMCS Stripe Billing Google Camera ImgIX Netlify Google Keep SocialPilot Hootsuite Firebase Analytics Access Manager Wordle Amazon Redshift Snowflake Microsoft Active Directory ClubHouse Tenable Nessus Obsidian Scrivener IDA Neo4j Pushbullet Pushover TinkerCAD Fusion360 SolidWorks TablePlus Cryptomator Glasswire Comodo Firewall Coyim Splunk Hungry Bring Panther IFTTT openHAB Alexa Google Home Twitch Asana IBM Watson Discovery FL Studio Ableton Google Maps Gather Aseprite Instagram Agora Wowza Docuware ELO Office Apollo GraphQL Supabase Hasura Stepzen Postgraphile Lyket.dev Kahoot Clubdesk Fairgate Bandicam Revoltchat Element Imply Pinot MongoDB Oracle Peoplesoft CurseForge Google Tag Manager MS SQL AppWrite Nhost AWS Kendra QnA Maker Apigee Google Cloud IoT Core Microsoft OneNote Amazon API Gateway Qualtrics Sprig Hotjar Sibelius Finale Dorico Snyk Common Room Orbit Toggl Track Adobe Scan Microsoft Lens CamScanner Vercel Stack Overflow Traktor Pro 3 Markup CMS Documentation Atlassian Confluence Raindrop Akeneo Salsify Informatica SuiteCRM VtigerCRM Cruise Tesla autopilot Waymo Adobe Animate Pencil2D Men&Mice Solarwinds Infoblox Device42 AWS WAF
Zilliz's Towhee

Zilliz's Towhee

Open Source Alternative to AWS SageMaker
Language
Python
Stars
3364
Watchers
3364
Forks
258
Open Issues
1
Last Updated
5/4/2025

REAMDE.md

 

x2vec, Towhee is all you need!

ENGLISH | 中文文档

 

Towhee is a cutting-edge framework designed to streamline the processing of unstructured data through the use of Large Language Model (LLM) based pipeline orchestration. It is uniquely positioned to extract invaluable insights from diverse unstructured data types, including lengthy text, images, audio and video files. Leveraging the capabilities of generative AI and the SOTA deep learning models, Towhee is capable of transforming this unprocessed data into specific formats such as text, image, or embeddings. These can then be efficiently loaded into an appropriate storage system like a vector database. Developers can initially build an intuitive data processing pipeline prototype with user friendly Pythonic API, then optimize it for production environments.

🎨 Multi Modalities: Towhee is capable of handling a wide range of data types. Whether it's image data, video clips, text, audio files, or even molecular structures, Towhee can process them all.

📃 LLM Pipeline orchestration: Towhee offers flexibility to adapt to different Large Language Models (LLMs). Additionally, it allows for hosting open-source large models locally. Moreover, Towhee provides features like prompt management and knowledge retrieval, making the interaction with these LLMs more efficient and effective.

🎓 Rich Operators: Towhee provides a wide range of ready-to-use state-of-the-art models across five domains: CV, NLP, multimodal, audio, and medical. With over 140 models like BERT and CLIP and rich functionalities like video decoding, audio slicing, frame sampling, and dimensionality reduction, it assists in efficiently building data processing pipelines.

🔌 Prebuilt ETL Pipelines: Towhee offers ready-to-use ETL (Extract, Transform, Load) pipelines for common tasks such as Retrieval-Augmented Generation, Text Image search, and Video copy detection. This means you don't need to be an AI expert to build applications using these features. ⚡️ High performance backend: Leveraging the power of the Triton Inference Server, Towhee can speed up model serving on both CPU and GPU using platforms like TensorRT, Pytorch, and ONNX. Moreover, you can transform your Python pipeline into a high-performance docker container with just a few lines of code, enabling efficient deployment and scaling.

🐍 Pythonic API: Towhee includes a Pythonic method-chaining API for describing custom data processing pipelines. We also support schemas, which makes processing unstructured data as easy as handling tabular data.

Getting started

Towhee requires Python 3.7+. You can install Towhee via pip:

pip install towhee towhee.models

Pipeline

Pre-defined Pipeline

Towhee provides some pre-defined pipelines to help users quickly implement some functions. Currently implemented are:

All pipelines can be found on Towhee Hub. Here is an example of using the sentence_embedding pipeline:

from towhee import AutoPipes, AutoConfig
# get the built-in sentence_similarity pipeline
config = AutoConfig.load_config('sentence_embedding')
config.model = 'paraphrase-albert-small-v2'
config.device = 0
sentence_embedding = AutoPipes.pipeline('sentence_embedding', config=config)

# generate embedding for one sentence
embedding = sentence_embedding('how are you?').get()
# batch generate embeddings for multi-sentences
embeddings = sentence_embedding.batch(['how are you?', 'how old are you?'])
embeddings = [e.get() for e in embeddings]

Custom pipelines

If you can't find the pipeline you want in towhee hub, you can also implement custom pipelines through the towhee Python API. In the following example, we will create a cross-modal retrieval pipeline based on CLIP.


from towhee import ops, pipe, DataCollection
# create image embeddings and build index
p = (
    pipe.input('file_name')
    .map('file_name', 'img', ops.image_decode.cv2())
    .map('img', 'vec', ops.image_text_embedding.clip(model_name='clip_vit_base_patch32', modality='image'))
    .map('vec', 'vec', ops.towhee.np_normalize())
    .map(('vec', 'file_name'), (), ops.ann_insert.faiss_index('./faiss', 512))
    .output()
)

for f_name in ['https://raw.githubusercontent.com/towhee-io/towhee/main/assets/dog1.png',
               'https://raw.githubusercontent.com/towhee-io/towhee/main/assets/dog2.png',
               'https://raw.githubusercontent.com/towhee-io/towhee/main/assets/dog3.png']:
    p(f_name)

# Flush faiss data into disk. 
p.flush()
# search image by text
decode = ops.image_decode.cv2('rgb')
p = (
    pipe.input('text')
    .map('text', 'vec', ops.image_text_embedding.clip(model_name='clip_vit_base_patch32', modality='text'))
    .map('vec', 'vec', ops.towhee.np_normalize())
    # faiss op result format:  [[id, score, [file_name], ...]
    .map('vec', 'row', ops.ann_search.faiss_index('./faiss', 3))
    .map('row', 'images', lambda x: [decode(item[2][0]) for item in x])
    .output('text', 'images')
)

DataCollection(p('puppy Corgi')).show()

Core Concepts

Towhee is composed of four main building blocks - Operators, Pipelines, DataCollection API and Engine.

  • Operators: An operator is a single building block of a neural data processing pipeline. Different implementations of operators are categorized by tasks, with each task having a standard interface. An operator can be a deep learning model, a data processing method, or a Python function.

  • Pipelines: A pipeline is composed of several operators interconnected in the form of a DAG (directed acyclic graph). This DAG can direct complex functionalities, such as embedding feature extraction, data tagging, and cross modal data analysis.

  • DataCollection API: A Pythonic and method-chaining style API for building custom pipelines, providing multiple data conversion interfaces: map, filter, flat_map, concat, window, time_window, and window_all. Through these interfaces, complex data processing pipelines can be built quickly to process unstructured data such as video, audio, text, images, etc.

  • Engine: The engine sits at Towhee's core. Given a pipeline, the engine will drive dataflow among individual operators, schedule tasks, and monitor compute resource usage (CPU/GPU/etc). We provide a basic engine within Towhee to run pipelines on a single-instance machine and a Triton-based engine for docker containers.

Resource

  • TowheeHub: https://towhee.io/
  • docs: https://towhee.readthedocs.io/en/latest/
  • examples: https://github.com/towhee-io/examples

Contributing

Writing code is not the only way to contribute! Submitting issues, answering questions, and improving documentation are just some of the many ways you can help our growing community. Check out our contributing page for more information.

Special thanks goes to these folks for contributing to Towhee, either on Github, our Towhee Hub, or elsewhere:




Looking for a database to store and index your embedding vectors? Check out Milvus.

Categories:
ML Ops