b4tman/sync_ics2gcal
1
0
Fork 0
mirror of https://github.com/b4tman/sync_ics2gcal synced 2025-05-15 21:34:31 +00:00
Code Issues Releases Activity

Merge branch 'develop'

Browse Source
This commit is contained in:
Dmitry Belyaev 2021-05-01 18:11:56 +03:00
commit 06f3776b1a
Signed by: b4tman
GPG Key ID: 41A00BF15EA7E5F3
23 changed files with 1212 additions and 851 deletions

1
.git_archival.txt Normal file

@ -0,0 +1 @@
ref-names: $Format:%D$

1
.gitattributes vendored Normal file

@ -0,0 +1 @@
.git_archival.txt export-subst

9
.github/dependabot.yml vendored Normal file

@ -0,0 +1,9 @@
version: 2
updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: monthly
time: '02:00'
open-pull-requests-limit: 10
target-branch: develop

38
.github/workflows/codeql-analysis.yml vendored Normal file

@ -0,0 +1,38 @@
name: "CodeQL"
on:
push:
branches: [develop, ]
pull_request:
# The branches below must be a subset of the branches above
branches: [develop]
schedule:
- cron: '0 12 10 * *'
jobs:
analyse:
name: Analyse
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
# Override language selection by uncommenting this and choosing your languages
with:
languages: python
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1

2
.github/workflows/pythonpackage.yml vendored

@ -17,7 +17,7 @@ jobs:
strategy:
max-parallel: 4
matrix:
python-version: [3.5, 3.6, 3.7, 3.8]
python-version: [3.6, 3.7, 3.8, 3.9]
steps:
- uses: actions/checkout@v1

26
.github/workflows/pythonpublish.yml vendored Normal file

@ -0,0 +1,26 @@
name: Upload Python Package
on:
release:
types: [created]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools setuptools_scm setuptools_scm_git_archive wheel twine
- name: Build and publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.pypi_token }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*

8
.gitignore vendored

@ -2,4 +2,10 @@ config.yml
service-account.json
*.pyc
my-test*.ics
.vscode/*
.vscode/
.idea/
/dist/
/*.egg-info/
/build/
/.eggs/
venv/

2
.travis.yml

@ -1,10 +1,10 @@
language: python
python:
- "3.5"
- "3.6"
- "3.7"
- "3.8"
- "3.9"
script:
- pytest -v

7
MANIFEST.in Normal file

@ -0,0 +1,7 @@
include pyproject.toml
# Include the README
include *.md
# Include the license file
include LICENSE

91
README.md

@ -1,15 +1,96 @@
# sync_ics2gcal
[![PyPI version](https://badge.fury.io/py/sync-ics2gcal.svg)](https://badge.fury.io/py/sync-ics2gcal)
[![Build Status](https://travis-ci.org/b4tman/sync_ics2gcal.svg?branch=master)](https://travis-ci.org/b4tman/sync_ics2gcal)
[![Dependabot Status](https://api.dependabot.com/badges/status?host=github&repo=b4tman/sync_ics2gcal)](https://dependabot.com) [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fb4tman%2Fsync_ics2gcal.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fb4tman%2Fsync_ics2gcal?ref=badge_shield)
![Python package status](https://github.com/b4tman/sync_ics2gcal/workflows/Python%20package/badge.svg)
Python scripts for sync .ics file with Google calendar
## Installation
To install from [PyPI](https://pypi.org/project/sync-ics2gcal/) with [pip](https://pypi.python.org/pypi/pip), run:
```sh
pip install sync-ics2gcal
```
Or download source code and install:
```sh
python setup.py install
```
## Configuration
### Create application in Google API Console
1. Create a new project: [console.developers.google.com/project](https://console.developers.google.com/project)
2. Choose the new project from the top right project dropdown (only if another project is selected)
3. In the project Dashboard, choose "Library"
4. Find and Enable "Google Calendar API"
5. In the project Dashboard, choose "Credentials"
6. In the "Service Accounts" group, click to "Manage service accounts"
7. Click "Create service account"
8. Choose service account name and ID
9. Go back to "Service Accounts" group in "Credentials"
10. Edit service account and click "Create key", choose JSON and download key file.
### Create working directory
For example: `/home/user/myfolder`.
1. Save service account key in file `service-account.json`.
2. Download [sample config](https://github.com/b4tman/sync_ics2gcal/blob/develop/sample-config.yml) and save to file `config.yml`. For example:
```sh
wget https://raw.githubusercontent.com/b4tman/sync_ics2gcal/develop/sample-config.yml -O config.yml
```
3. *(Optional)* Place source `.ics` file, `my-calendar.ics` for example.
### Configuration parameters
* `start_from` - start date:
* full format datetime, `2018-04-03T13:23:25.000001Z` for example
* or just `now`
* *(Optional)* `service_account` - service account filename, remove it from config to use [default credentials](https://developers.google.com/identity/protocols/application-default-credentials)
* *(Optional)* `logging` - [config](https://docs.python.org/3.8/library/logging.config.html#dictionary-schema-details) to setup logging
* `google_id` - target google calendar id, `my-calendar@group.calendar.google.com` for example
* `source` - source `.ics` filename, `my-calendar.ics` for example
## Usage
### Manage calendars
```sh
manage-ics2gcal GROUP | COMMAND
```
**GROUPS**:
* **property** - get/set properties (see [CalendarList resource](https://developers.google.com/calendar/v3/reference/calendarList#resource)), subcommands:
- **get** - get calendar property
- **set** - set calendar property
**COMMANDS**:
* **list** - list calendars
* **create** - create calendar
* **add_owner** - add owner to calendar
* **remove** - remove calendar
* **rename** - rename calendar
Use **-h** for more info.
### Sync calendar
just type:
```sh
sync-ics2gcal
```
## How it works
![How it works](how-it-works.png)
## License
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fb4tman%2Fsync_ics2gcal.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fb4tman%2Fsync_ics2gcal?ref=badge_large)

104
manage-calendars.py

@ -1,104 +0,0 @@
import argparse
import datetime
import logging.config
import yaml
from pytz import utc
from gcal_sync import GoogleCalendar, GoogleCalendarService
def parse_args():
parser = argparse.ArgumentParser(
description="manage google calendars in service account")
command_subparsers = parser.add_subparsers(help='command', dest='command')
command_subparsers.add_parser('list', help='list calendars')
parser_create = command_subparsers.add_parser(
'create', help='create calendar')
parser_create.add_argument(
'summary', action='store', help='new calendar summary')
parser_create.add_argument('--timezone', action='store',
default=None, required=False, help='new calendar timezone')
parser_create.add_argument(
'--public', default=False, action='store_true', help='make calendar public')
parser_add_owner = command_subparsers.add_parser(
'add_owner', help='add owner to calendar')
parser_add_owner.add_argument('id', action='store', help='calendar id')
parser_add_owner.add_argument(
'owner_email', action='store', help='new owner email')
parser_remove = command_subparsers.add_parser(
'remove', help='remove calendar')
parser_remove.add_argument(
'id', action='store', help='calendar id to remove')
parser_rename = command_subparsers.add_parser(
'rename', help='rename calendar')
parser_rename.add_argument(
'id', action='store', help='calendar id')
parser_rename.add_argument(
'summary', action='store', help='new summary')
args = parser.parse_args()
if args.command is None:
parser.print_usage()
return args
def load_config():
with open('config.yml', 'r', encoding='utf-8') as f:
result = yaml.safe_load(f)
return result
def list_calendars(service):
response = service.calendarList().list(fields='items(id,summary)').execute()
for calendar in response.get('items'):
print('{summary}: {id}'.format_map(calendar))
def create_calendar(service, summary, timezone, public):
calendar = GoogleCalendar(service, None)
calendar.create(summary, timezone)
if public:
calendar.make_public()
print('{}: {}'.format(summary, calendar.calendarId))
def add_owner(service, id, owner_email):
calendar = GoogleCalendar(service, id)
calendar.add_owner(owner_email)
print('to {} added owner: {}'.format(id, owner_email))
def remove_calendar(service, id):
calendar = GoogleCalendar(service, id)
calendar.delete()
print('removed: {}'.format(id))
def rename_calendar(service, id, summary):
calendar = {'summary': summary}
service.calendars().patch(body=calendar, calendarId=id).execute()
print('{}: {}'.format(summary, id))
def main():
args = parse_args()
config = load_config()
if 'logging' in config:
logging.config.dictConfig(config['logging'])
srv_acc_file = config['service_account']
service = GoogleCalendarService.from_srv_acc_file(srv_acc_file)
if 'list' == args.command:
list_calendars(service)
elif 'create' == args.command:
create_calendar(service, args.summary, args.timezone, args.public)
elif 'add_owner' == args.command:
add_owner(service, args.id, args.owner_email)
elif 'remove' == args.command:
remove_calendar(service, args.id)
elif 'rename' == args.command:
rename_calendar(service, args.id, args.summary)
if __name__ == '__main__':
main()

3
pyproject.toml Normal file

@ -0,0 +1,3 @@
[build-system]
requires = ["setuptools>=40.8.0", "wheel", "setuptools_scm"]
build-backend = "setuptools.build_meta"

11
requirements.txt

@ -1,5 +1,6 @@
google-auth==1.11.0
google-api-python-client==1.7.11
icalendar==4.0.4
pytz==2019.3
PyYAML==5.3
google-auth==1.30.0
google-api-python-client==2.3.0
icalendar==4.0.7
pytz==2021.1
PyYAML==5.4.1
fire==0.4.0

7
setup.cfg Normal file

@ -0,0 +1,7 @@
[metadata]
license_files = LICENSE
[options]
setup_requires =
setuptools_scm
setuptools_scm_git_archive

44
setup.py Normal file

@ -0,0 +1,44 @@
import setuptools
with open('README.md', 'r') as fh:
long_description = fh.read()
setuptools.setup(
name='sync-ics2gcal',
author='Dmitry Belyaev',
author_email='b4tm4n@mail.ru',
license='MIT',
description='Sync ics file with Google calendar',
long_description=long_description,
long_description_content_type='text/markdown',
url='https://github.com/b4tman/sync_ics2gcal',
use_scm_version={
'fallback_version': '0.1',
'local_scheme': 'no-local-version'
},
setup_requires=['setuptools_scm', 'setuptools_scm_git_archive'],
packages=setuptools.find_packages(exclude=['tests']),
classifiers=[
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
],
python_requires='>=3.6',
install_requires = [
'google-auth>=1.5.0',
'google-api-python-client>=1.7.0',
'icalendar>=4.0.1',
'pytz',
'PyYAML>=3.13'
],
entry_points={
"console_scripts": [
"sync-ics2gcal = sync_ics2gcal.sync_calendar:main",
"manage-ics2gcal = sync_ics2gcal.manage_calendars:main",
]
}
)

0
gcal_sync/__init__.py → sync_ics2gcal/__init__.py

138
gcal_sync/gcal.py → sync_ics2gcal/gcal.py

@ -1,13 +1,14 @@
import logging
import sys
import google.auth
from google.oauth2 import service_account
from googleapiclient import discovery
from pytz import utc
from datetime import datetime
from typing import List, Dict, Any, Callable, Tuple, Optional
class GoogleCalendarService():
class GoogleCalendarService:
"""class for make google calendar service Resource
Returns:
@ -15,20 +16,53 @@ class GoogleCalendarService():
"""
@staticmethod
def from_srv_acc_file(service_account_file):
"""make service Resource from service account filename (authorize)
Returns:
service Resource
def default():
"""make service Resource from default credentials (authorize)
( https://developers.google.com/identity/protocols/application-default-credentials )
( https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#google.auth.default )
"""
scopes = ['https://www.googleapis.com/auth/calendar']
credentials = service_account.Credentials.from_service_account_file(service_account_file)
scoped_credentials = credentials.with_scopes(scopes)
service = discovery.build('calendar', 'v3', credentials=scoped_credentials)
credentials, _ = google.auth.default(scopes=scopes)
service = discovery.build(
'calendar', 'v3', credentials=credentials, cache_discovery=False)
return service
def select_event_key(event):
@staticmethod
def from_srv_acc_file(service_account_file: str):
"""make service Resource from service account filename (authorize)
"""
scopes = ['https://www.googleapis.com/auth/calendar']
credentials = service_account.Credentials.from_service_account_file(
service_account_file)
scoped_credentials = credentials.with_scopes(scopes)
service = discovery.build(
'calendar', 'v3', credentials=scoped_credentials,
cache_discovery=False)
return service
@staticmethod
def from_config(config: Optional[Dict[str, Optional[str]]] = None):
"""make service Resource from config dict
Arguments:
config -- config with keys:
(optional) service_account: - service account filename
if key not in dict then default credentials will be used
( https://developers.google.com/identity/protocols/application-default-credentials )
-- None: default credentials will be used
"""
if config is not None and 'service_account' in config:
service = GoogleCalendarService.from_srv_acc_file(
config['service_account'])
else:
service = GoogleCalendarService.default()
return service
def select_event_key(event: Dict[str, Any]) -> Optional[str]:
"""select event key for logging
Arguments:
@ -46,17 +80,17 @@ def select_event_key(event):
return key
class GoogleCalendar():
class GoogleCalendar:
"""class to interact with calendar on google
"""
logger = logging.getLogger('GoogleCalendar')
def __init__(self, service, calendarId):
self.service = service
self.calendarId = calendarId
def __init__(self, service: discovery.Resource, calendarId: Optional[str]):
self.service: discovery.Resource = service
self.calendarId: str = calendarId
def _make_request_callback(self, action, events_by_req):
def _make_request_callback(self, action: str, events_by_req: List[Dict[str, Any]]) -> Callable:
"""make callback for log result of batch request
Arguments:
@ -72,8 +106,10 @@ class GoogleCalendar():
key = select_event_key(event)
if exception is not None:
self.logger.error('failed to %s event with %s: %s, exception: %s',
action, key, event.get(key), str(exception))
self.logger.error(
'failed to %s event with %s: %s, exception: %s',
action, key, event.get(key), str(exception)
)
else:
resp_key = select_event_key(response)
if resp_key is not None:
@ -83,7 +119,7 @@ class GoogleCalendar():
action, key, event.get(key))
return callback
def list_events_from(self, start):
def list_events_from(self, start: datetime) -> List[Dict[str, Any]]:
""" list events from calendar, where start date >= start
"""
fields = 'nextPageToken,items(id,iCalUID,updated)'
@ -92,8 +128,11 @@ class GoogleCalendar():
timeMin = utc.normalize(start.astimezone(utc)).replace(
tzinfo=None).isoformat() + 'Z'
while True:
response = self.service.events().list(calendarId=self.calendarId, pageToken=page_token,
singleEvents=True, timeMin=timeMin, fields=fields).execute()
response = self.service.events().list(calendarId=self.calendarId,
pageToken=page_token,
singleEvents=True,
timeMin=timeMin,
fields=fields).execute()
if 'items' in response:
events.extend(response['items'])
page_token = response.get('nextPageToken')
@ -102,7 +141,7 @@ class GoogleCalendar():
self.logger.info('%d events listed', len(events))
return events
def find_exists(self, events):
def find_exists(self, events: List) -> Tuple[List[Tuple[Dict[str, Any], Dict[str, Any]]], List[Dict[str, Any]]]:
""" find existing events from list, by 'iCalUID' field
Arguments:
@ -120,15 +159,16 @@ class GoogleCalendar():
def list_callback(request_id, response, exception):
found = False
event = events_by_req[int(request_id)]
cur_event = events_by_req[int(request_id)]
if exception is None:
found = ([] != response['items'])
else:
self.logger.error('exception %s, while listing event with UID: %s', str(
exception), event['iCalUID'])
self.logger.error(
'exception %s, while listing event with UID: %s',
str(exception), cur_event['iCalUID'])
if found:
exists.append(
(event, response['items'][0]))
(cur_event, response['items'][0]))
else:
not_found.append(events_by_req[int(request_id)])
@ -137,14 +177,19 @@ class GoogleCalendar():
for event in events:
events_by_req.append(event)
batch.add(self.service.events().list(calendarId=self.calendarId,
iCalUID=event['iCalUID'], showDeleted=True, fields=fields), request_id=str(i))
iCalUID=event['iCalUID'],
showDeleted=True,
fields=fields
),
request_id=str(i)
)
i += 1
batch.execute()
self.logger.info('%d events exists, %d not found',
len(exists), len(not_found))
return exists, not_found
def insert_events(self, events):
def insert_events(self, events: List[Dict[str, Any]]):
""" insert list of events
Arguments:
@ -160,11 +205,13 @@ class GoogleCalendar():
for event in events:
events_by_req.append(event)
batch.add(self.service.events().insert(
calendarId=self.calendarId, body=event, fields=fields), request_id=str(i))
calendarId=self.calendarId, body=event, fields=fields),
request_id=str(i)
)
i += 1
batch.execute()
def patch_events(self, event_tuples):
def patch_events(self, event_tuples: List[Tuple[Dict[str, Any], Dict[str, Any]]]):
""" patch (update) events
Arguments:
@ -182,11 +229,12 @@ class GoogleCalendar():
continue
events_by_req.append(event_new)
batch.add(self.service.events().patch(
calendarId=self.calendarId, eventId=event_old['id'], body=event_new), fields=fields, request_id=str(i))
calendarId=self.calendarId, eventId=event_old['id'],
body=event_new), fields=fields, request_id=str(i))
i += 1
batch.execute()
def update_events(self, event_tuples):
def update_events(self, event_tuples: List[Tuple[Dict[str, Any], Dict[str, Any]]]):
""" update events
Arguments:
@ -204,11 +252,12 @@ class GoogleCalendar():
continue
events_by_req.append(event_new)
batch.add(self.service.events().update(
calendarId=self.calendarId, eventId=event_old['id'], body=event_new, fields=fields), request_id=str(i))
calendarId=self.calendarId, eventId=event_old['id'],
body=event_new, fields=fields), request_id=str(i))
i += 1
batch.execute()
def delete_events(self, events):
def delete_events(self, events: List[Dict[str, Any]]):
""" delete events
Arguments:
@ -223,11 +272,12 @@ class GoogleCalendar():
for event in events:
events_by_req.append(event)
batch.add(self.service.events().delete(
calendarId=self.calendarId, eventId=event['id']), request_id=str(i))
calendarId=self.calendarId,
eventId=event['id']), request_id=str(i))
i += 1
batch.execute()
def create(self, summary, timeZone=None):
def create(self, summary: str, timeZone: Optional[str] = None) -> Any:
"""create calendar
Arguments:
@ -244,7 +294,9 @@ class GoogleCalendar():
if timeZone is not None:
calendar['timeZone'] = timeZone
created_calendar = self.service.calendars().insert(body=calendar).execute()
created_calendar = self.service.calendars().insert(
body=calendar
).execute()
self.calendarId = created_calendar['id']
return created_calendar
@ -264,9 +316,12 @@ class GoogleCalendar():
},
'role': 'reader'
}
return self.service.acl().insert(calendarId=self.calendarId, body=rule_public).execute()
return self.service.acl().insert(
calendarId=self.calendarId,
body=rule_public
).execute()
def add_owner(self, email):
def add_owner(self, email: str):
"""add calendar owner by email
Arguments:
@ -280,4 +335,7 @@ class GoogleCalendar():
},
'role': 'owner'
}
return self.service.acl().insert(calendarId=self.calendarId, body=rule_owner).execute()
return self.service.acl().insert(
calendarId=self.calendarId,
body=rule_owner
).execute()

59
gcal_sync/ical.py → sync_ics2gcal/ical.py

@ -1,12 +1,14 @@
import datetime
import logging
from typing import Union, Dict, Any, Callable, Optional, List
from icalendar import Calendar, Event
from pytz import utc
def format_datetime_utc(value):
def format_datetime_utc(value: Union[datetime.date, datetime.datetime]) -> str:
"""utc datetime as string from date or datetime value
Arguments:
value -- date or datetime value
@ -17,23 +19,29 @@ def format_datetime_utc(value):
value = datetime.datetime(
value.year, value.month, value.day, tzinfo=utc)
value = value.replace(microsecond=1)
return utc.normalize(value.astimezone(utc)).replace(tzinfo=None).isoformat() + 'Z'
return utc.normalize(
value.astimezone(utc)
).replace(tzinfo=None).isoformat() + 'Z'
def gcal_date_or_dateTime(value, check_value=None):
def gcal_date_or_dateTime(value: Union[datetime.date, datetime.datetime],
check_value: Union[datetime.date, datetime.datetime, None] = None)\
-> Dict[str, str]:
"""date or dateTime to gcal (start or end dict)
Arguments:
value -- date or datetime value
check_value - date or datetime to choise result type (if not None)
value: date or datetime
check_value: optional for choose result type
Returns:
dict { 'date': ... } or { 'dateTime': ... }
{ 'date': ... } or { 'dateTime': ... }
"""
if check_value is None:
check_value = value
result = {}
result: Dict[str, str] = {}
if isinstance(check_value, datetime.datetime):
result['dateTime'] = format_datetime_utc(value)
else:
@ -49,7 +57,7 @@ class EventConverter(Event):
( https://developers.google.com/calendar/v3/reference/events#resource-representations )
"""
def _str_prop(self, prop):
def _str_prop(self, prop: str) -> str:
"""decoded string property
Arguments:
@ -61,7 +69,7 @@ class EventConverter(Event):
return self.decoded(prop).decode(encoding='utf-8')
def _datetime_str_prop(self, prop):
def _datetime_str_prop(self, prop: str) -> str:
"""utc datetime as string from property
Arguments:
@ -73,7 +81,7 @@ class EventConverter(Event):
return format_datetime_utc(self.decoded(prop))
def _gcal_start(self):
def _gcal_start(self) -> Dict[str, str]:
""" event start dict from icalendar event
Raises:
@ -86,7 +94,7 @@ class EventConverter(Event):
value = self.decoded('DTSTART')
return gcal_date_or_dateTime(value)
def _gcal_end(self):
def _gcal_end(self) -> Dict[str, str]:
"""event end dict from icalendar event
Raises:
@ -109,7 +117,9 @@ class EventConverter(Event):
raise ValueError('no DTEND or DURATION')
return result
def _put_to_gcal(self, gcal_event, prop, func, ics_prop=None):
def _put_to_gcal(self, gcal_event: Dict[str, Any],
prop: str, func: Callable[[str], str],
ics_prop: Optional[str] = None):
"""get property from ical event if exist, and put to gcal event
Arguments:
@ -124,7 +134,7 @@ class EventConverter(Event):
if ics_prop in self:
gcal_event[prop] = func(ics_prop)
def to_gcal(self):
def to_gcal(self) -> Dict[str, Any]:
"""Convert
Returns:
@ -132,12 +142,11 @@ class EventConverter(Event):
"""
event = {
'iCalUID': self._str_prop('UID')
'iCalUID': self._str_prop('UID'),
'start': self._gcal_start(),
'end': self._gcal_end()
}
event['start'] = self._gcal_start()
event['end'] = self._gcal_end()
self._put_to_gcal(event, 'summary', self._str_prop)
self._put_to_gcal(event, 'description', self._str_prop)
self._put_to_gcal(event, 'location', self._str_prop)
@ -145,33 +154,35 @@ class EventConverter(Event):
self._put_to_gcal(
event, 'updated', self._datetime_str_prop, 'LAST-MODIFIED')
self._put_to_gcal(
event, 'transparency', lambda prop: self._str_prop(prop).lower(), 'TRANSP')
event,
'transparency',
lambda prop: self._str_prop(prop).lower(), 'TRANSP')
return event
class CalendarConverter():
class CalendarConverter:
"""Convert icalendar events to google calendar resources
"""
logger = logging.getLogger('CalendarConverter')
def __init__(self, calendar=None):
self.calendar = calendar
def __init__(self, calendar: Optional[Calendar] = None):
self.calendar: Optional[Calendar] = calendar
def load(self, filename):
def load(self, filename: str):
""" load calendar from ics file
"""
with open(filename, 'r', encoding='utf-8') as f:
self.calendar = Calendar.from_ical(f.read())
self.logger.info('%s loaded', filename)
def loads(self, string):
def loads(self, string: str):
""" load calendar from ics string
"""
self.calendar = Calendar.from_ical(string)
def events_to_gcal(self):
def events_to_gcal(self) -> List[Dict[str, Any]]:
"""Convert events to google calendar resources
"""

144
sync_ics2gcal/manage_calendars.py Normal file

@ -0,0 +1,144 @@
import logging.config
from typing import Optional, Dict, Any, List
import fire
import yaml
from . import GoogleCalendar, GoogleCalendarService
def load_config(filename: str) -> Optional[Dict[str, Any]]:
result = None
try:
with open(filename, 'r', encoding='utf-8') as f:
result = yaml.safe_load(f)
except FileNotFoundError:
pass
return result
class PropertyCommands:
""" get/set google calendar properties """
def __init__(self, _service):
self._service = _service
def get(self, calendar_id: str, property_name: str) -> None:
""" get calendar property
Args:
calendar_id: calendar id
property_name: property key
"""
response = self._service.calendarList().get(calendarId=calendar_id,
fields=property_name).execute()
print(response.get(property_name))
def set(self, calendar_id: str, property_name: str, property_value: str) -> None:
""" set calendar property
Args:
calendar_id: calendar id
property_name: property key
property_value: property value
"""
body = {property_name: property_value}
response = self._service.calendarList().patch(body=body, calendarId=calendar_id).execute()
print(response)
class Commands:
""" manage google calendars in service account """
def __init__(self, config: str = 'config.yml'):
"""
Args:
config(str): config filename
"""
self._config: Optional[Dict[str, Any]] = load_config(config)
if self._config is not None and 'logging' in self._config:
logging.config.dictConfig(self._config['logging'])
self._service = GoogleCalendarService.from_config(self._config)
self.property = PropertyCommands(self._service)
def list(self, show_hidden: bool = False, show_deleted: bool = False) -> None:
""" list calendars
Args:
show_hidden: show hidden calendars
show_deleted: show deleted calendars
"""
fields: str = 'nextPageToken,items(id,summary)'
calendars: List[Dict[str, Any]] = []
page_token: Optional[str] = None
while True:
calendars_api = self._service.calendarList()
response = calendars_api.list(fields=fields,
pageToken=page_token,
showHidden=show_hidden,
showDeleted=show_deleted
).execute()
if 'items' in response:
calendars.extend(response['items'])
page_token = response.get('nextPageToken')
if page_token is None:
break
for calendar in calendars:
print('{summary}: {id}'.format_map(calendar))
def create(self, summary: str, timezone: Optional[str] = None, public: bool = False) -> None:
""" create calendar
Args:
summary: new calendar summary
timezone: new calendar timezone
public: make calendar public
"""
calendar = GoogleCalendar(self._service, None)
calendar.create(summary, timezone)
if public:
calendar.make_public()
print('{}: {}'.format(summary, calendar.calendarId))
def add_owner(self, calendar_id: str, email: str) -> None:
""" add owner to calendar
Args:
calendar_id: calendar id
email: new owner email
"""
calendar = GoogleCalendar(self._service, calendar_id)
calendar.add_owner(email)
print('to {} added owner: {}'.format(calendar_id, email))
def remove(self, calendar_id: str) -> None:
""" remove calendar
Args:
calendar_id: calendar id
"""
calendar = GoogleCalendar(self._service, calendar_id)
calendar.delete()
print('removed: {}'.format(calendar_id))
def rename(self, calendar_id: str, summary: str) -> None:
""" rename calendar
Args:
calendar_id: calendar id
summary:
"""
calendar = {'summary': summary}
self._service.calendars().patch(body=calendar, calendarId=calendar_id).execute()
print('{}: {}'.format(summary, calendar_id))
def main():
fire.Fire(Commands, name='manage-ics2gcal')
if __name__ == '__main__':
main()

86
gcal_sync/sync.py → sync_ics2gcal/sync.py

@ -1,22 +1,33 @@
import datetime
import dateutil.parser
import logging
import operator
from typing import List, Any, Dict, Set, Tuple, Union, Callable
import dateutil.parser
from pytz import utc
from .gcal import GoogleCalendar
from .ical import CalendarConverter
class CalendarSync():
class CalendarSync:
"""class for syncronize calendar with google
"""
logger = logging.getLogger('CalendarSync')
def __init__(self, gcalendar, converter):
self.gcalendar = gcalendar
self.converter = converter
def __init__(self, gcalendar: GoogleCalendar, converter: CalendarConverter):
self.gcalendar: GoogleCalendar = gcalendar
self.converter: CalendarConverter = converter
self.to_insert: List[Dict[str, Any]] = []
self.to_update: List[Tuple[Dict[str, Any], Dict[str, Any]]] = []
self.to_delete: List[Dict[str, Any]] = []
@staticmethod
def _events_list_compare(items_src, items_dst, key='iCalUID'):
def _events_list_compare(items_src: List[Dict[str, Any]],
items_dst: List[Dict[str, Any]],
key: str = 'iCalUID') \
-> Tuple[List[Dict[str, Any]], List[Tuple[Dict[str, Any], Dict[str, Any]]], List[Dict[str, Any]]]:
""" compare list of events by key
Arguments:
@ -30,16 +41,18 @@ class CalendarSync():
items_to_delete)
"""
def get_key(item): return item[key]
def get_key(item: Dict[str, Any]) -> str: return item[key]
keys_src = set(map(get_key, items_src))
keys_dst = set(map(get_key, items_dst))
keys_src: Set[str] = set(map(get_key, items_src))
keys_dst: Set[str] = set(map(get_key, items_dst))
keys_to_insert = keys_src - keys_dst
keys_to_update = keys_src & keys_dst
keys_to_delete = keys_dst - keys_src
def items_by_keys(items, key_name, keys):
def items_by_keys(items: List[Dict[str, Any]],
key_name: str,
keys: Set[str]) -> List[Dict[str, Any]]:
return list(filter(lambda item: item[key_name] in keys, items))
items_to_insert = items_by_keys(items_src, key, keys_to_insert)
@ -57,14 +70,19 @@ class CalendarSync():
""" filter 'to_update' events by 'updated' datetime
"""
def filter_updated(event_tuple):
def filter_updated(event_tuple: Tuple[Dict[str, Any], Dict[str, Any]]) -> bool:
new, old = event_tuple
return dateutil.parser.parse(new['updated']) > dateutil.parser.parse(old['updated'])
new_date = dateutil.parser.parse(new['updated'])
old_date = dateutil.parser.parse(old['updated'])
return new_date > old_date
self.to_update = list(filter(filter_updated, self.to_update))
@staticmethod
def _filter_events_by_date(events, date, op):
def _filter_events_by_date(events: List[Dict[str, Any]],
date: Union[datetime.date, datetime.datetime],
op: Callable[[Union[datetime.date, datetime.datetime],
Union[datetime.date, datetime.datetime]], bool]) -> List[Dict[str, Any]]:
""" filter events by start datetime
Arguments:
@ -76,10 +94,10 @@ class CalendarSync():
list of filtred events
"""
def filter_by_date(event):
def filter_by_date(event: Dict[str, Any]) -> bool:
date_cmp = date
event_start = event['start']
event_date = None
event_start: Dict[str, str] = event['start']
event_date: Union[datetime.date, datetime.datetime, str, None] = None
compare_dates = False
if 'date' in event_start:
@ -91,14 +109,15 @@ class CalendarSync():
event_date = dateutil.parser.parse(event_date)
if compare_dates:
date_cmp = datetime.date(date.year, date.month, date.day)
event_date = datetime.date(event_date.year, event_date.month, event_date.day)
event_date = datetime.date(
event_date.year, event_date.month, event_date.day)
return op(event_date, date_cmp)
return list(filter(filter_by_date, events))
@staticmethod
def _tz_aware_datetime(date):
def _tz_aware_datetime(date: Union[datetime.date, datetime.datetime]) -> datetime.datetime:
"""make tz aware datetime from datetime/date (utc if no tzinfo)
Arguments:
@ -114,7 +133,7 @@ class CalendarSync():
date = date.replace(tzinfo=utc)
return date
def prepare_sync(self, start_date):
def prepare_sync(self, start_date: Union[datetime.date, datetime.datetime]) -> None:
"""prepare sync lists by comparsion of events
Arguments:
@ -132,35 +151,38 @@ class CalendarSync():
events_src_past = CalendarSync._filter_events_by_date(
events_src, start_date, operator.lt)
events_src = None
# first events comparsion
self.to_insert, self.to_update, self.to_delete = CalendarSync._events_list_compare(
events_src_pending, events_dst)
events_src_pending, events_dst = None, None
# find in events 'to_delete' past events from source, for update (move to past)
_, add_to_update, self.to_delete = CalendarSync._events_list_compare(
events_src_past, self.to_delete)
self.to_update.extend(add_to_update)
events_src_past = None
# find if events 'to_insert' exists in gcalendar, for update them
add_to_update, self.to_insert = self.gcalendar.find_exists(
self.to_insert)
self.to_update.extend(add_to_update)
add_to_update = None
# exclude outdated events from 'to_update' list, by 'updated' field
self._filter_events_to_update()
self.logger.info('prepared to sync: ( insert: %d, update: %d, delete: %d )',
len(self.to_insert), len(self.to_update), len(self.to_delete))
self.logger.info(
'prepared to sync: ( insert: %d, update: %d, delete: %d )',
len(self.to_insert),
len(self.to_update),
len(self.to_delete)
)
def apply(self):
def clear(self) -> None:
""" clear prepared sync lists (insert, update, delete)
"""
self.to_insert.clear()
self.to_update.clear()
self.to_delete.clear()
def apply(self) -> None:
""" apply sync (insert, update, delete), using prepared lists of events
"""
@ -168,6 +190,6 @@ class CalendarSync():
self.gcalendar.update_events(self.to_update)
self.gcalendar.delete_events(self.to_delete)
self.logger.info('sync done')
self.clear()
self.to_insert, self.to_update, self.to_delete = [], [], []
self.logger.info('sync done')

17
sync-calendar.py → sync_ics2gcal/sync_calendar.py

@ -1,10 +1,12 @@
from typing import Dict, Any
import yaml
import dateutil.parser
import datetime
import logging
import logging.config
from gcal_sync import (
from . import (
CalendarConverter,
GoogleCalendarService,
GoogleCalendar,
@ -12,14 +14,13 @@ from gcal_sync import (
)
def load_config():
def load_config() -> Dict[str, Any]:
with open('config.yml', 'r', encoding='utf-8') as f:
result = yaml.safe_load(f)
return result
def get_start_date(date_str):
result = datetime.datetime(1,1,1)
def get_start_date(date_str: str) -> datetime.datetime:
if 'now' == date_str:
result = datetime.datetime.utcnow()
else:
@ -33,21 +34,21 @@ def main():
if 'logging' in config:
logging.config.dictConfig(config['logging'])
calendarId = config['calendar']['google_id']
ics_filepath = config['calendar']['source']
srv_acc_file = config['service_account']
calendarId: str = config['calendar']['google_id']
ics_filepath: str = config['calendar']['source']
start = get_start_date(config['start_from'])
converter = CalendarConverter()
converter.load(ics_filepath)
service = GoogleCalendarService.from_srv_acc_file(srv_acc_file)
service = GoogleCalendarService.from_config(config)
gcalendar = GoogleCalendar(service, calendarId)
sync = CalendarSync(gcalendar, converter)
sync.prepare_sync(start)
sync.apply()
if __name__ == '__main__':
main()

10
tests/test_converter.py

@ -1,6 +1,8 @@
from typing import Tuple
import pytest
from gcal_sync import CalendarConverter
from sync_ics2gcal import CalendarConverter
uid = "UID:uisgtr8tre93wewe0yr8wqy@test.com"
only_start_date = uid + """
@ -26,11 +28,11 @@ LAST-MODIFIED:20180326T120235Z
"""
def ics_test_cal(content):
def ics_test_cal(content: str) -> str:
return "BEGIN:VCALENDAR\r\n{}END:VCALENDAR\r\n".format(content)
def ics_test_event(content):
def ics_test_event(content: str) -> str:
return ics_test_cal("BEGIN:VEVENT\r\n{}END:VEVENT\r\n".format(content))
@ -68,7 +70,7 @@ def param_events_start_end(request):
return request.param
def test_event_start_end(param_events_start_end):
def test_event_start_end(param_events_start_end: Tuple[str, str, str, str]):
(date_type, ics_str, start, end) = param_events_start_end
converter = CalendarConverter()
converter.loads(ics_str)

47
tests/test_sync.py

@ -3,15 +3,16 @@ import hashlib
import operator
from copy import deepcopy
from random import shuffle
from typing import Union, List, Dict, Optional
import dateutil.parser
import pytest
from pytz import timezone, utc
from gcal_sync import CalendarSync
from sync_ics2gcal import CalendarSync
def sha1(string):
def sha1(string: Union[str, bytes]) -> str:
if isinstance(string, str):
string = string.encode('utf8')
h = hashlib.sha1()
@ -19,55 +20,57 @@ def sha1(string):
return h.hexdigest()
def gen_events(start, stop, start_time, no_time=False):
def gen_events(start: int,
stop: int,
start_time: Union[datetime.datetime, datetime.date],
no_time: bool = False) -> List[Dict[str, Union[str, Dict[str, str]]]]:
if no_time:
start_time = datetime.date(
start_time.year, start_time.month, start_time.day)
duration = datetime.date(1, 1, 2) - datetime.date(1, 1, 1)
date_key = "date"
suff = ''
duration: datetime.timedelta = datetime.date(1, 1, 2) - datetime.date(1, 1, 1)
date_key: str = "date"
date_end: str = ''
else:
start_time = utc.normalize(
start_time.astimezone(utc)).replace(tzinfo=None)
duration = datetime.datetime(
1, 1, 1, 2) - datetime.datetime(1, 1, 1, 1)
date_key = "dateTime"
suff = 'Z'
duration: datetime.timedelta = datetime.datetime(1, 1, 1, 2) - datetime.datetime(1, 1, 1, 1)
date_key: str = "dateTime"
date_end: str = 'Z'
result = []
result: List[Dict[str, Union[str, Dict[str, str]]]] = []
for i in range(start, stop):
event_start = start_time + (duration * i)
event_end = event_start + duration
updated = event_start
updated: Union[datetime.datetime, datetime.date] = event_start
if no_time:
updated = datetime.datetime(
updated.year, updated.month, updated.day, 0, 0, 0, 1, tzinfo=utc)
event = {
event: Dict[str, Union[str, Dict[str, str]]] = {
'summary': 'test event __ {}'.format(i),
'location': 'la la la {}'.format(i),
'description': 'test TEST -- test event {}'.format(i),
"iCalUID": "{}@test.com".format(sha1("test - event {}".format(i))),
"updated": updated.isoformat() + 'Z',
"created": updated.isoformat() + 'Z'
"created": updated.isoformat() + 'Z',
'start': {date_key: event_start.isoformat() + date_end},
'end': {date_key: event_end.isoformat() + date_end}
}
event['start'] = {date_key: event_start.isoformat() + suff}
event['end'] = {date_key: event_end.isoformat() + suff}
result.append(event)
return result
def gen_list_to_compare(start, stop):
result = []
def gen_list_to_compare(start: int, stop: int) -> List[Dict[str, str]]:
result: List[Dict[str, str]] = []
for i in range(start, stop):
result.append({'iCalUID': 'test{:06d}'.format(i)})
return result
def get_start_date(event):
event_start = event['start']
start_date = None
def get_start_date(event: Dict[str, Union[str, Dict[str, str]]]) -> Union[datetime.datetime, datetime.date]:
event_start: Dict[str, str] = event['start']
start_date: Optional[str] = None
is_date = False
if 'date' in event_start:
start_date = event_start['date']
@ -113,7 +116,7 @@ def test_compare():
@pytest.mark.parametrize("no_time", [True, False], ids=['date', 'dateTime'])
def test_filter_events_by_date(no_time):
def test_filter_events_by_date(no_time: bool):
msk = timezone('Europe/Moscow')
now = utc.localize(datetime.datetime.utcnow())
msk_now = msk.normalize(now.astimezone(msk))