diff --git a/Corbia spider.md b/Corbia spider.md index 8c93f61..0928e59 100644 --- a/Corbia spider.md +++ b/Corbia spider.md @@ -1,2 +1,4 @@ -In order to index the web properly, corbia spider only \ No newline at end of file +In order to index the web properly, corbia spider uses domain based templates. + +For each domain, a custom python function will be used in order to extract the relevant information and store it as JSON before further processing. \ No newline at end of file