Need help with url-tracker?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

ahussam
210 Stars 39 Forks 38 Commits 9 Opened issues

Description

Change monitoring app that checks the content of web pages in different periods.

Services available

!
?

Need anything else?

Contributors list

# 50,632
PHP
monitor...
certifi...
Shell
1 commit

URL Tracker v 0.3.0

Build Status License Maintenance GitHub issues

Change monitoring app that checks the content of web pages in different periods (hourly, daily, weekly, monthly) and detects if they were modified since the last check. It can be used to monitor S3, Azure, JS files, ...etc.

Motivation :battery:

  • Blog Post: https://ahussam.me/careem-aws-s3-bucket-takeover/

How to use :arrow_forward:

  • If you want to track changes on a web page click add new link button on the right side (You must be logged in):

    • Fill description textbox with any text that describes the target
      e.g: Uber internal system login page
      .
    • Write the target link in Link of page textbox
      e.g: http://s3.amazonaws.com/careemcrm/
      .
    • Write keywords that will be tracked on the page if they appear you will get noticed
      e.g: 404, new feature, not found
      Note: They MUST NOT be there in the first request.
    • If you want to track an authenticated page add your cookies in the cookie text box.
    • Select the peroid that you want to check the page at.
    • In tolerance set the accepted differences between the previous fetch and the next one by characters
      e.g: 100
      . Note: if you want to ingore minors changes like CSRF tokens, cache tokens, cookie tracking set this parameter. In case you leave it blank the server will ship 2 requests and comapres the differences between them and sets the differences (acceptedChange) in DB.
    • Then click add link button. You should see the new target in the URLs list.
  • If you want to get notifications via email:

    • Create a gmail account.
    • Go to
      User Control > Settings
      then set the gmail account that will send the notificaiton (DON'T USE YOUR PRIVATE EMAIL).
    • Write your private email in the receiver textbox.
    • Click save changes button.

Features :ballotboxwith_check:

  • Login system.
  • 4 periods checks.
  • Email reporting.
  • Search in items.
  • Dynamic pages fetching.
  • Authenticated fetching.
  • Diff checker
  • Bot reporting (Telegram, Slack, ...etc) SOON!
  • Suggest a feature!

Code Organisation :openfilefolder:

+---api // Controllers, models, helpers 
+---assets // UI assets 
+---config // Config files: routes, security, datastores, ...etc
+---crontab // Fetch function file
+---scripts // Cloud SDK
+---tasks // Grunt tasks
+---view 
       \---layout // App layout
        \---pages // HTML in ejs templates 

Technologies :hammerandwrench:

Backend :gear: :

| Name | Description | Link | ------ | ------ | ------ | Nodejs | Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.| https://nodejs.org/en/ Sails.js | Sails.js makes it easy to build custom, enterprise-grade Node.js apps. | https://sailsjs.com/ MongoDB | MongoDB is a cross-platform document-oriented database program. |https://www.mongodb.com/ EJS| EJS is a simple templating language that lets you generate HTML markup with plain JavaScript. |https://ejs.co/| ...|...|...|

Frontend :desktop_computer:

| Name | Description | Link | ------ | ------ | ------ | Vue.js| The Progressive JavaScript Framework. | https://vuejs.org | Bootstrap | The most popular HTML, CSS, and JS library in the world. | https://getbootstrap.com/ Font Awesome|The world's most popular and easiest to use icon.|https://fontawesome.com/| ...|...|...|

Object Diagram :cardfilebox:

OD


REST API :link:

| URL | Description | ------ | ------ | /api/v1/account/* | Account operations /api/v1/settings | Settings operations /api/v1/link/* | Target operations /api/v1/entrance/login| Login operation


Deployment :rocket:

Note: make sure to install mongodb on your OS.

[email protected]:~# git clone https://github.com/ahussam/url-tracker.git
[email protected]:~# cd url-tracker
[email protected]:~/url-tracker# npm install 
[email protected]:~/url-tracker# nodejs app.js 

Then go to http://127.0.0.1:1337

For deployment on a live server install MongoDB then replace the lines 50 & 51 in

config/env/production.js
with:
      adapter: 'sails-mongo',
      url: 'mongodb://localhost/urlTracker', 

For more information check this out: https://sailsjs.com/documentation/concepts/deployment/hosting

Docker Option :whale2:

[email protected]:~# git clone https://github.com/ahussam/url-tracker.git
[email protected]:~# cd url-tracker
[email protected]:~/url-tracker# docker-compose build
[email protected]:~/url-tracker# docker-compose up

Then go to http://127.0.0.1:8080


Default Credential :key:

PLEASE MODIFIY THEM AFTER SIGN IN FOR THE FIRST TIME.

Email | Password ------ | -------

[email protected]| 123456

Screenshots :camera:

ss

ss

ss

ss

ss


License :pagefacingup:

The MIT License (MIT)

Copyright (c) 2020 Abdullah Hussam

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.