The first public repository that provides free BUBT website scraping API script on Github.

Overview

BUBT WEBSITE SCRAPPING SCRIPT

I think this is the first public repository that provides free BUBT website scraping API script on github. When I was doing my 3rd year project one for my friend Abdullah Xayed wrote an web scraping project for me. Now I add some extra feature on it.

Important Note

I am not sharing some api script with you due to security reason. And I am requisting you not to use any of this provided api for production use. I already give you the API script. So, Host them on your web server and then use them for the production.

Project Directories

Path Description
./cources Contains all course scrapping scripts.
./faculties/scripts Contains all facultie scrapping scripts.
./notices_events Contains notices_events scrapping scripts.
./faculties Contains all faculties image by department name sub folder.
./json_data Contains all the courses, faculties & notices_events information in json format.

API Contribution

  • API BY BUBT

     •Student Verify 
     •Faculty Verify
    
  • Abdullah Xayed

     •Annex Login 
     •Annex Result 
     •Annex Fees 
     •Annex Routine 
     •All Events 
     •Event Details 
     •All Notices 
     •Notice Details 
    
  • Md. Imam Hossain

     •Courses API Scrit: CSE, EEE, CIVIL, TEXTILE, BBA, ECO, EDE, LLB, ENG.
     •Faculty API Script: CSE, EEE, CIVIL, TEXTILE, BBA, ECO, EDE, LLB, ENG.
     •Notice 
     •Notice Details 
     •Events 
     •Events Details 
     •Routine API
    

API Response & Type

BUBT API:

Name Method Description Examples
Student Verify GET Verify bubt students /global_file/getData.php?id=?&type=?
Faculty Verify GET Verify bubt faculty /global_file/getData.php?id=?&type=?

Abdullah Xayed API: (v1)

Name Method Description Examples
Annex Login GET Verify bubt faculty /bubt/v1/login?id=?&pass=?
Annex Result GET Get student result from annex by session id /bubt/v1/prevCourses?phpsessid=?
Annex Fees GET Get student fees from annex by session id /bubt/v1/fees?phpsessid=?
Annex Routine GET Get student routine from annex by session id Not working, Cause routine shift from annex to BUBT Soft /bubt/v1/routine?phpsessid=?
All Events GET Get all events from bubt website /bubt/v1/allEvent?
Events Details GET Get an event details by events url /bubt/v1/eventDetails?url=?
All Notice GET Get all notices from bubt website /bubt/v1/allNotice?
Notice Details GET Get a notice details by notices url /bubt/v1/noticeDetails?url=?

Md. Imam Hossain API: (v2)

Name Method Description Examples
Events GET Get events from bubt website by page and limit /bubt/v2/allEvent?page=?&limit=?
Events Details GET Get an event details by events url bubt/v2/eventDetails?url=?
Notice GET Get notices from bubt website by page and limit /bubt/v2/allNotice?page=?&limit=?
Notice Details GET Get a notice details by notices url /bubt/v2/noticeDetails?url=?
Smart Routine GET Get student routine from BUBT Soft by students id Secret API /bubt/v2/getRoutine?stdId=?

Sample Json Data

BUBT API:

Student Verify:

{
  "sis_std_id": "17181103084",
  "sis_std_name": "Md. Imam Hossain",
  "sis_std_prgrm_sn": "B.Sc. Engg. in CSE",
  "sis_std_prgrm_id": "006",
  "sis_std_intk": "37",
  "sis_std_email": "[email protected]",
  "sis_std_father": "Mahbub Rashid",
  "sis_std_gender": "M",
  "sis_std_LocGuardian": "Mahbub Rashid",
  "sis_std_Bplace": "Vasantek, Dhaka",
  "sis_std_Status": "R",
  "sis_std_blood": "",
  "gazo": "data:image/jpeg;base64,"
}

Faculty Verify:

[
  {
    "EmpId": "18020331033",
    "DemoId": "18020331033",
    "EmpName": "Md. Ahsanul Haque",
    "DOB": "1996-06-21T00:00:00",
    "PermanentAddress": "South Atapara, Bogura Sadar-5800, Bogura",
    "FatherName": "Md. Abdul Awal",
    "ECName": "Md. Abdul Awal",
    "ECNo": "01711936404",
    "ECRelation": "Father",
    "Gender": "Male",
    "DeptName": "Department of Computer Science & Engineering",
    "PosName": "Lecturer",
    "BloodGroup": "A+",
    "StatusId": "1",
    "EmpImage": "data:image/jpeg;base64,"
    }
]

Abdullah Xayed API:(v1)

Annex Login:

{
  "PHPSESSID": "7d1755fe6c32b74d321fe3d3ba69a4ad",
  "status": "success"
}

Annex Result:

{
  "data": [
    {
      "cgpa": "3.22",
      "results": [
        {
          "code": "ENG 101",
          "credit": "3",
          "grade": "B-",
          "title": "English Language-I",
          "type": "Theory"
        }
      ],
      "semester": "Fall, 2017-18",
      "sgpa": "3.22"
    }
  ],
  "status": "success"
}

Annex Fees:

{
  "data": [
    {
      "Demand": "44195",
      "Due": "0",
      "Paid": "44195",
      "Remarks": "Semester Charge+Tuition Fees+Others",
      "Semester": "Fall, 2017-18",
      "Waiver": "0",
      "payments": [
        {
          "Account_Code": "319",
          "Payment_Amount": "15600",
          "Payment_No": "1",
          "Reciept_No": "18888",
          "Waiver": "0"
        },
        {
          "Account_Code": "319",
          "Payment_Amount": "28595",
          "Payment_No": "2",
          "Reciept_No": "43019",
          "Waiver": "0"
        }
      ]
    }
  ],
  "result": {
    "Total_Demand": "384816",
    "Total_Due": "7442",
    "Total_Paid": "353923",
    "Total_Waiver": "23451"
  },
  "status": "success"
}

Annex Routine:

{
  "data": [
    {
      "Building": "",
      "Day": "Saturday",
      "Intake": "",
      "Room_No": "",
      "Schedule": "08:30 AM to 10:00 AM",
      "Section": "",
      "Subject_Code": "",
      "Teacher_Code": ""
    }
  ],
  "status": "success"
}

All Events:

{
  "data": [
    {
      "published_on": "5 Aug 2021",
      "title": "International Conference on Science and Contemporary Technologies (ICSCT) Opened at BUBT",
      "url": "https://www.bubt.edu.bd/home/event_details/200"
    }
  ],
  "status": "success"
}

Annex Notices:

{
  "data": [
      {
        "category": "Exam Related",
        "published_on": "8 Oct 2021",
        "title": "Defense Notice",
        "url": "https://www.bubt.edu.bd/home/notice_details/665"
      }
  ],
  "status": "success"
}

Events Details:

{
    "data": {
      "description": "Bangladesh University of  Business and Technology  (BUBT) organized a virtual Orientation  Program for Spring 2021 Students on April 22, 2021....",
      "downloads": [
        {
          "url": ""
        }
      ],
      "images": [
        {
          "url": "https://www.bubt.edu.bd/assets/frontend/media/1619504011BUBT_22_04__2021.jpg"
        }
      ],
      "pubDate": "25 Apr 2021",
      "title": "Virtual Orientation for Spring 2021 Students at BUBT"
    },
    "status": "success"
  }

Notice Details:

{
    "data": {
      "description": "Defense Notice\nThis is to notify the intern students that their Online Internship Defense will be held in Google Meet...",
      "downloads": [
        {
          "url": ""
        }
      ],
      "images": [
        {
          "url": ""
        }
      ],
      "pubDate": "8 Oct 2021",
      "title": "Defense Notice"
    },
    "status": "success"
}

Md. Imam Hossain API:(v2)

Events:

{
  "data": [
    {
      "category": "Event",
      "details": {
        "description": "Honorable Vice Chancellor, Professor Dr. Muhammed Fayyaz Khan inaugurated the Fire Drill at BUBT...",
        "images": "https://www.bubt.edu.bd/assets/frontend/media/1623580519IMG_8259.JPG"
      },
      "id": 199,
      "published_on": "13 Jun 2021",
      "title": "Fire Drill Training at BUBT",
      "url": "https://www.bubt.edu.bd/home/event_details/199"
    }
  ],
  "status": "success",
  "type": "event"
}

Events Details:

{
  "data": {
    "description": "The 2nd international conference on Science and Contemporary Technologies (ICSCT) arranged by BUBT officially opened on August 05, 2021 in Dhaka, attended by nearly 200 researchers from more than 20 different countries of the world...",
    "images": "https://www.bubt.edu.bd/assets/frontend/media/16281864231.jpg"
  }
}

Notice:

{
  "data": [
    {
      "category": "Exam Related",
      "details": {
        "description": "Defense Notice\nThis is to notify the intern students that their Online Internship Defense will be held in Google Meet..",
        "images": ""
      },
      "id": 665,
      "published_on": "8 Oct 2021",
      "title": "Defense Notice",
      "url": "https://www.bubt.edu.bd/home/notice_details/665"
    }
  ],
  "status": "success",
  "type": "notice"
}

Notice Details:

{
  "data": {
    "description": "Defense Notice\nThis is to notify the intern students that their Online Internship Defense will be held in Google Meet...",
    "images": ""
  }
}

Smart Routine:

{
  "PHPSESSID": "7d1755fe6c32b74d321fe3d3ba69a4ad",
  "status": "success"
}

Courses:

{
    "data":[
       {
          "course_code":"CSE 101",
          "course_title":"Computer and Programming Concepts",
          "theory_credit":"3.00",
          "lab_credit":"1.50",
          "total_credit":"4.50",
          "prerequisit":""
       }
    ]
}

Faculties:

{
    "data":[
       {
        "teacher_name": "Dr.Muhammad Firoz Mridha",
        "teacher_code": "DMFM",
        "email": "[email protected]",
        "designation": "Associate Professor & Chairman",
        "status": "",
        "link": "https://www.bubt.edu.bd/department/member_details/332",
        "image": "https://storage.googleapis.com/bubt-smart-routine.appspot.com/cse/Dr.Muhammad%20Firoz%20Mridha_3199.png"
      }
    ]
}

🧑 Author

Md. Imam Hossain

You can also follow my GitHub Profile to stay updated about my latest projects:

GitHub Follow

If you liked the repo then kindly support it by giving it a star !

Copyright (c) 2021 MD. IMAM HOSSAIN

Owner
Md Imam Hossain
Lazy coder.
Md Imam Hossain
This is a web crawler that works on employ email data by gmane.org and visualizes it in different ways.

crawler_to_visual_gmane Analyzing an EMAIL Archive from gmane and vizualizing the data using the D3 JavaScript library. This is a set of tools that al

Saim Zafar 1 Dec 20, 2021
Minecraft Item Scraper

Minecraft Item Scraper To run, first ensure you have the BeautifulSoup module: pip install bs4 Then run, python minecraft_items.py folder-to-save-ima

Jaedan Calder 1 Dec 29, 2021
An Web Scraping API for MDL(My Drama List) for Python.

PyMDL An API for MyDramaList(MDL) based on webscraping for python. Description An API for MDL to make your life easier in retriving and working on dat

6 Dec 10, 2022
Scraping Top Repositories for Topics on GitHub,

0.-Webscrapping-using-python Scraping Top Repositories for Topics on GitHub, Web scraping is the process of extracting and parsing data from websites

Dev Aravind D Satprem 2 Mar 18, 2022
河南工业大学 完美校园 自动校外打卡

HAUT-checkin 河南工业大学自动校外打卡 由于github actions存在明显延迟,建议直接使用腾讯云函数 特点 多人打卡 使用简单,仅需账号密码以及用于微信推送的uid 自动获取上一次打卡信息用于打卡 向所有成员微信单独推送打卡状态 完美校园服务器繁忙时造成打卡失败会自动重新打卡

36 Oct 27, 2022
A Web Scraper built with beautiful soup, that fetches udemy course information. Get udemy course information and convert it to json, csv or xml file

Udemy Scraper A Web Scraper built with beautiful soup, that fetches udemy course information. Installation Virtual Environment Firstly, it is recommen

Aditya Gupta 15 May 17, 2022
Incredibly fast crawler designed for OSINT.

Photon Incredibly fast crawler designed for OSINT. Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap Key Features Dat

Somdev Sangwan 9.3k Jan 02, 2023
A Powerful Spider(Web Crawler) System in Python.

pyspider A Powerful Spider(Web Crawler) System in Python. Write script in Python Powerful WebUI with script editor, task monitor, project manager and

Roy Binux 15.7k Jan 04, 2023
Simple library for exploring/scraping the web or testing a website you’re developing

Robox is a simple library with a clean interface for exploring/scraping the web or testing a website you’re developing. Robox can fetch a page, click on links and buttons, and fill out and submit for

Dan Claudiu Pop 79 Nov 27, 2022
哔哩哔哩爬取器:以个人为中心

Open Bilibili Crawer 哔哩哔哩是一个信息非常丰富的社交平台,我们基于此构造社交网络。在该网络中,节点包括用户(up主),以及视频、专栏等创作产物;关系包括:用户之间,包括关注关系(following/follower),回复关系(评论区),转发关系(对视频or动态转发);用户对创

Boshen Shi 3 Oct 21, 2021
Scraping weather data using Python to receive umbrella reminders

A Python package which scrapes weather data from google and sends umbrella reminders to specified email at specified time daily.

Edula Vinay Kumar Reddy 1 Aug 23, 2022
Scrapy-based cyber security news finder

Cyber-Security-News-Scraper Scrapy-based cyber security news finder Goal To keep up to date on the constant barrage of information within the field of

2 Nov 01, 2021
让中国用户使用git从github下载的速度提高1000倍!

序言 github上有很多好项目,但是国内用户连github却非常的慢.每次都要用插件或者其他工具来解决. 这次自己做一个小工具,输入github原地址后,就可以自动替换为代理地址,方便大家更快速的下载. 安装 pip install cit 主要功能与用法 主要功能 change 将目标地址转换为

35 Aug 29, 2022
Dailyiptvlist.com Scraper With Python

Dailyiptvlist.com scraper Info Made in python Linux only script Script requires to have wget installed Running script Clone repository with: git clone

1 Oct 16, 2021
An IpVanish Proxies Scraper

EzProxies Tired of searching for good proxies for hours? Just get an IpVanish account and get thousands of good proxies in few seconds! Showcase Watch

11 Nov 13, 2022
Snowflake database loading utility with Scrapy integration

Snowflake Stage Exporter Snowflake database loading utility with Scrapy integration. Meant for streaming ingestion of JSON serializable objects into S

Oleg T. 0 Dec 06, 2021
A web scraping pipeline project that retrieves TV and movie data from two sources, then transforms and stores data in a MySQL database.

New to Streaming Scraper An in-progress web scraping project built with Python, R, and SQL. The scraped data are movie and TV show information. The go

Charles Dungy 1 Mar 28, 2022
🐞 Douban Movie / Douban Book Scarpy

Python3-based Douban Movie/Douban Book Scarpy crawler for cover downloading + data crawling + review entry.

Xingbo Jia 1 Dec 03, 2022
Html Content / Article Extractor, web scrapping lib in Python

Python-Goose - Article Extractor Intro Goose was originally an article extractor written in Java that has most recently (Aug2011) been converted to a

Xavier Grangier 3.8k Jan 02, 2023
Scrapegoat is a python library that can be used to scrape the websites from internet based on the relevance of the given topic irrespective of language using Natural Language Processing

Scrapegoat is a python library that can be used to scrape the websites from internet based on the relevance of the given topic irrespective of language using Natural Language Processing. It can be ma

10 Jul 06, 2022