Skip to content

Owo Sugiana / iso8583-web

Go to a project
Commit f6fd2281 by Owo Sugiana
Options

Web service ke ISO8583 dijalankan pserve

1 parent 4f72dce1
Inline Side-by-side
Showing 54 changed files with 934 additions and 2645 deletions
CHANGES.txt
0.4 2020-11-15
--------------
- Daemon bin/iso8583 tidak lagi memuat web service. Sebagai gantinya pembuatan
web service seperti biasa di direktori iso8583_web/views/. bin/pserve bisa
memuat thread ISO8583 bila di konfigurasi ada section berawalan host_,
section yang sama seperti yang digunakan bin/iso8583. bin/iso8583 cocok
sebagai daemon Pemda yang selama ini tidak butuh web service. Namun begitu ke
depan bin/pserve dirancang sebagai webmin seperti untuk mengubah konfigurasi,
restart thread ISO8583 saat sepi transaksi, melihat log, dll. Dalam kasus
LinkAja - PBB yang merupakan web service to ISO8583 service maka keduanya
terhubung oleh file iso8583_web/common.py.
- Web service maupun web client jsonrpc dan linkaja dipindahkan ke paket
opensipkd-iso8583-bjb.
0.3.4 2020-10-06
----------------
- Validasi prefix Invoice ID PBB LinkAja
......
MANIFEST.in
include CHANGES.txt production.ini development.ini README.rst
recursive-include iso8583_web *.py *.png *.css *.pt *.tpl *.mako
recursive-include iso8583_web *.py *.png *.css *.pt *.tpl *.mako *.pot *.po *.mo *.csv
README.rst
......@@ -17,7 +17,7 @@ yang dibutuhkan `ISO8583 <https://pypi.org/project/ISO8583/>`_::
Pemasangan yang tidak otomatis ini agar tidak menimbulkan kegagalan. Lanjut::
$ ~/env/bin/pip install git+https://git.opensipkd.com/sugiana/iso8583-web
$ ~/env/bin/pip install git+https://git.opensipkd.com/sugiana/iso8583-web.git
Selanjutnya kita membutuhkan tiga terminal untuk simulasi:
......@@ -35,7 +35,7 @@ Berikut ini diagram komunikasi ISO8583 Pemda dengan BJB::
| BJB port 10002 | <-----> | Pemda |
+----------------+ +-------+
Buat file konfigurasi ``test-pemda.ini``::
Buat file konfigurasi ``h2h-pemda.ini``::
[loggers]
keys = root, iso8583_web
......@@ -71,7 +71,7 @@ Buat file konfigurasi ``test-pemda.ini``::
format = %(asctime)s %(levelname)s %(message)s
[module_opensipkd.iso8583.bjb.pbb.bogor_kota]
db_url = postgresql://user:pass@localhost/db
db_url = postgresql://user:pass@localhost/db_bogor_kota
persen_denda = 2
nip_pencatat = 999999999
kd_kanwil = 01
......@@ -105,14 +105,14 @@ section, misalnya ``host_mitracomm``.
Modul ``opensipkd.iso8583.bjb.pbb.bogor_kota`` tidak ada di paket ini sehingga
kita perlu memasangnya::
$ ~/env/bin/pip install git+https://git.opensipkd.com/sugiana/opensipkd-iso8583-bjb
$ ~/env/bin/pip install git+https://git.opensipkd.com/sugiana/opensipkd-iso8583-bjb.git
Modul ini memerlukan tabel untuk mencatat transaksi. Terlebih dahulu buat file
konfigurasi ``iso8583-pbb.ini`` yang merupakan *non daemon*::
[main]
module = opensipkd.iso8583.bjb.pbb.bogor_kota
db_url = postgresql://user:pass@localhost/db
db_url = postgresql://user:pass@localhost/db_bogor_kota
Buat tabelnya::
......@@ -120,7 +120,7 @@ Buat tabelnya::
Pastikan tidak ada kesalahan. Lalu jalankan daemon-nya::
$ ~/env/bin/iso8583 test-pemda.ini
$ ~/env/bin/iso8583 h2h-pemda.ini
Anda akan mendapat pesan seperti ini::
......@@ -143,7 +143,41 @@ kantor pusatnya::
HTTP atau web service merupakan komunikasi satu arah, oleh karena itu hanya
digambarkan satu panah saja. Selanjutnya buka terminal ke-2, dan buat file
``test-bank.ini``::
``h2h-bank.ini``::
[app:main]
use = egg:iso8583_web
pyramid.reload_templates = true
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = id
pyramid.includes =
pyramid_debugtoolbar
pyramid_rpc.jsonrpc
opensipkd.views.bjb.jsonrpc
sqlalchemy.url = postgresql://user:pass@localhost/db_bank
sqlalchemy.pool_size = 50
sqlalchemy.max_overflow = 100
sqlalchemy.pool_pre_ping = true
timezone = Asia/Jakarta
localization = id_ID.UTF-8
mail.host = localhost
mail.port = 25
mail.username = user@example.com
mail.password = FIXME
mail.sender_name = Example Name
retry.attempts = 3
[server:main]
use = egg:waitress#main
listen = localhost:7000
threads = 12
[loggers]
keys = root, iso8583_web, jsonrpc
......@@ -189,73 +223,29 @@ digambarkan satu panah saja. Selanjutnya buka terminal ke-2, dan buat file
listen = true
streamer = bjb_with_suffix
module = opensipkd.iso8583.bjb.pbb.test
db_url = postgresql://user:pass@localhost/db_bank
[web]
port = 7000
threads = 12
[web_teller]
route_path = /rpc
host = kota_bogor
module = iso8583_web.scripts.views.jsonrpc
allowed_ip =
127.0.0.1
10.8.20.1
Perbedaannya adalah pada:
1. Log file
2. ``listen = true`` yang berarti bank sebagai server-nya
3. Penggunaan modul ``opensipkd.iso8583.bjb.pbb.test`` dimana modul ini
tidak menggunakan database sehingga tidak perlu konfigurasi tambahan.
Juga ada konfigurasi terkait web server yang merupakan gerbang aplikasi teller
untuk transaksi. Perhatikan section ``web`` ini::
[web]
port = 7000
threads = 12
dimana:
Buat dulu tabel terkait Pyramid::
* ``port`` adalah network port yang *listen*. Tentu saja yang dimaksud
adalah HTTP port.
* ``threads`` adalah jumlah *worker* yang siaga. Ini artinya pada saat yang
sama web server ini sanggup menangani 12 web client.
$ ~/env/bin/initialize_iso8583_web_db h2h-bank.ini
Selanjutnya section ``web_teller``::
Siapkan file konfigurasi ``iso8583-bank.ini`` sekedar untuk membuat tabel
terkait ISO8583::
[web_teller]
route_path = /rpc
host = kota_bogor
module = iso8583_web.scripts.views.jsonrpc
allowed_ip =
127.0.0.1
10.8.20.1
Section yang berawalan ``web_`` adalah konfigurasi untuk setiap jenis *web client*.
Bandingkan dengan section yang berawalan ``host_`` yang berarti konfigurasi
untuk ISO8583. Berikut penjelasannya:
[main]
module = opensipkd.iso8583.bjb.pbb.test
db_url = postgresql://user:pass@localhost/db_bank
* ``route_path`` merupakan *URL path*, selalu awali dengan ``/``
* ``host = kota_bogor`` berarti harus ada section ``host_kota_bogor``. Ini merupakan
*routing*, akan diarahkan ke ISO8583 mana HTTP request ini.
* ``module`` merupakan *modul Pyramid* yang akan digunakan untuk menanggapi
*HTTP request*
* ``allowed_ip`` berisi IP yang diperkenankan menggunakan jalur ini. Jika
dikosongkan maka diperkenankan diakses dari IP manapun.
Lalu buat tabelnya::
Bila konfigurasi web ini digabungkan maka URL-nya menjadi
``http://127.0.0.1:7000/rpc``.
$ ~/env/bin/iso8583_bjb_pbb_test_initdb iso8583-bank.ini
Sekarang jalankan::
$ ~/env/bin/iso8583 test-bank.ini
$ ~/env/bin/pserve test-bank.ini
Seharusnya ada penampakan seperti ini::
2020-07-18 22:27:44,360 INFO Web server JSON-RPC route path /rpc
2020-07-18 22:27:44,373 INFO Web server listen at 0.0.0.0:7000 with 12 workers
2020-07-18 22:27:44,373 INFO ISO8583 server listen at 0.0.0.0:10002
2020-07-18 22:27:46,592 INFO 127.0.0.1 allowed
2020-07-18 22:27:51,595 INFO 127.0.0.1 kota_bogor 140197249940784 Receive [b'00560800822000000000000004000000000000000718222751222236301\x03']
......@@ -394,9 +384,16 @@ berikut ini::
Meski ini hanya simulasi namun daemon bank membutuhkan database untuk pemetaan
STAN (bit 11) *payment request* dari Agratek dengan STAN saat ke Pemda. Informasi
ini digunakan saat *reversal request*. Kini buatlah tabelnya::
ini digunakan saat *reversal request*. Buatlah file konfigurasi
``initdb-test-aggregator.ini``` sekedar untuk proses init database::
[main]
module = opensipkd.iso8583.bjb.pbb.test_aggregator
db_url = postgresql://user:pass@localhost/db
Lalu buatlah tabelnya::
$ ~/env/bin/iso8583_bjb_pbb_test_aggregator_initdb
$ ~/env/bin/iso8583_bjb_pbb_test_aggregator_initdb initdb-test-aggregator.ini
Hidupkan lagi daemon-nya::
......
alembic.ini.tpl deleted 100644 → 0
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = alembic
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# timezone to use when rendering the date
# within the migration file as well as the filename.
# string value is passed to dateutil.tz.gettz()
# leave blank for localtime
# timezone =
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to alembic/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat alembic/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = {db_url}
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
contrib/bank-teller.py deleted 100644 → 0
import sys
import requests
import json
from optparse import OptionParser
from pprint import pprint
default_url = 'http://localhost:7000/rpc'
default_host = 'pemda'
help_url = 'default ' + default_url
help_host = 'default ' + default_host
parser = OptionParser()
parser.add_option('', '--url', default=default_url, help=help_url)
parser.add_option('', '--host', default=default_host, help=help_host)
option, args = parser.parse_args(sys.argv[1:])
url = option.url
headers = {'content-type': 'application/json'}
p = {'host': option.host}
data = {
'method': 'echo',
'params': [p],
'jsonrpc': '2.0',
'id': 0,
}
resp = requests.post(url, data=json.dumps(data), headers=headers)
json_resp = resp.json()
pprint(json_resp)
contrib/common.py deleted 100644 → 0
class CommonIso:
def get_label(self):
return (self.is_echo_request() and 'Echo Request') or \
(self.is_echo_response() and 'Echo Response') or \
(self.is_sign_on_request() or \
self.is_sign_on_response() or \
self.is_sign_off_request() or \
self.is_sign_off_response() or \
self.is_inquiry_request() or \
self.is_inquiry_response() or \
self.is_payment_request() or \
self.is_payment_response() or \
self.is_reversal_request() or \
self.is_reversal_response()
contrib/echo.py deleted 100644 → 0
import sys
from iso8583_web.read_conf import (
read_conf,
get_conf,
)
def show(iso):
flow = iso.is_request() and 'Request' or 'Response'
msg = '{} {} MTI {} Data {}'.format(
iso.get_name(), flow, iso.getMTI(), iso.get_values())
print(msg)
conf_file = sys.argv[1]
read_conf(conf_file)
ip = '127.0.0.1'
port = 10002
conf = get_conf(ip, port)
cls = conf['module_obj'].doc.Doc
iso_req = cls()
iso_req.echo_request()
show(iso_req)
iso_resp = cls(from_iso=iso_req)
iso_resp.process()
show(iso_resp)
iso_req = cls()
iso_req.sign_on_request()
show(iso_req)
iso_resp = cls(from_iso=iso_req)
iso_resp.process()
show(iso_resp)
iso_req = cls()
iso_req.sign_off_request()
show(iso_req)
iso_resp = cls(from_iso=iso_req)
iso_resp.process()
show(iso_resp)
contrib/inquiry.py deleted 100644 → 0
import sys
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from iso8583_web.read_conf import (
read_conf,
get_conf,
)
import sismiop.services
def show(iso):
flow = iso.is_request() and 'Request' or 'Response'
msg = '{} {} MTI {} Data {}'.format(
iso.get_name(), flow, iso.getMTI(), iso.get_values())
print(msg)
conf_file = sys.argv[1]
invoice_id = sys.argv[2]
read_conf(conf_file)
ip = '127.0.0.1'
port = 10002
conf = get_conf(ip, port)
engine = create_engine(conf['db_url'])
session_factory = sessionmaker(bind=engine)
sismiop.services.DBSession = session_factory()
cls = conf['module_obj'].doc.Doc
iso_req = cls(conf=conf)
iso_req.set_inquiry_request()
iso_req.set_invoice_id(invoice_id)
show(iso_req)
iso_resp = cls(from_iso=iso_req, conf=conf)
iso_resp.process()
show(iso_resp)
development.ini
......@@ -13,8 +13,11 @@ pyramid.debug_routematch = false
pyramid.default_locale_name = id
pyramid.includes =
pyramid_debugtoolbar
pyramid_rpc.jsonrpc
sqlalchemy.url = postgresql://user:pass@localhost/dbname
sqlalchemy.pool_pre_ping = true
timezone = Asia/Jakarta
localization = id_ID.UTF-8
......@@ -24,10 +27,18 @@ mail.username = user@example.com
mail.password = FIXME
mail.sender_name = Example Name
retry.attempts = 3
# By default, the toolbar only appears for clients from IP addresses
# '127.0.0.1' and '::1'.
# debugtoolbar.hosts = 127.0.0.1 ::1
[alembic]
# path to migration scripts
script_location = alembic
file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s
# file_template = %%(rev)s_%%(slug)s
###
# wsgi server configuration
###
......
iso8583-aggregator.ini
......@@ -3,65 +3,53 @@
# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###
[loggers]
keys = root, iso8583_web, jsonrpc
[app:main]
use = egg:iso8583_web
[handlers]
keys = console, file
pyramid.reload_templates = true
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = id
pyramid.includes =
opensipkd.views.bjb.linkaja
[formatters]
keys = generic
sqlalchemy.url = postgresql://user:pass@localhost/db
sqlalchemy.pool_pre_ping = true
[logger_root]
level = DEBUG
handlers = console, file
timezone = Asia/Jakarta
localization = id_ID.UTF-8
[logger_iso8583_web]
level = DEBUG
handlers =
qualname = iso8583_web
mail.host = localhost
mail.port = 25
mail.username = user@example.com
mail.password = FIXME
mail.sender_name = Example Name
[logger_jsonrpc]
level = DEBUG
handlers = console, file
qualname = pyramid_rpc.jsonrpc
retry.attempts = 3
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = DEBUG
formatter = generic
# By default, the toolbar only appears for clients from IP addresses
# '127.0.0.1' and '::1'.
# debugtoolbar.hosts = 127.0.0.1 ::1
[handler_file]
class = FileHandler
args = ('/home/sugiana/log/agratek.log', 'a')
level = DEBUG
formatter = generic
[alembic]
# path to migration scripts
script_location = iso8583_web/alembic
file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s
# file_template = %%(rev)s_%%(slug)s
[formatter_generic]
format = %(asctime)s %(levelname)s %(message)s
###
# wsgi server configuration
###
[web]
port = 7001
[server:main]
use = egg:waitress#main
listen = 0.0.0.0:7001
threads = 12
[web_linkaja_pbb]
route_path = /linkaja/pbb
module = iso8583_web.scripts.views.linkaja.pbb
db_url = postgresql://user:pass@localhost/db
host = bjb
timeout = 30
# Notification Message sesuai prefix Invoice ID
notification_message =
3271:PBB Kota Bogor
3676:PBB Kota Tangerang Selatan
3275:PBB Kota Bekasi
[web_linkaja_sambat]
route_path = /linkaja/sambat
module = iso8583_web.scripts.views.linkaja.sambat
db_url = postgresql://user:pass@localhost/db
bank_url = http://localhost:8080/sam-core-app2/rest/SAMWebService
###
# ISO-8583 Threads
###
[host_bjb]
ip = 127.0.0.1
......@@ -88,3 +76,45 @@ bit_60 =
3271:123
3676:142
3275:120
###
# Log
###
[loggers]
keys = root, iso8583_web, jsonrpc
[handlers]
keys = console, file
[formatters]
keys = generic
[logger_root]
level = DEBUG
handlers = console, file
[logger_iso8583_web]
level = DEBUG
handlers =
qualname = iso8583_web
[logger_jsonrpc]
level = DEBUG
handlers = console, file
qualname = pyramid_rpc.jsonrpc
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = DEBUG
formatter = generic
[handler_file]
class = FileHandler
args = ('/home/sugiana/log/agratek.log', 'a')
level = DEBUG
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)s %(message)s
iso8583-bank.ini
###
# app configuration
# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/environment.html
###
[app:main]
use = egg:iso8583_web
pyramid.reload_templates = true
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = id
pyramid.includes =
pyramid_debugtoolbar
pyramid_rpc.jsonrpc
opensipkd.views.bjb.jsonrpc
sqlalchemy.url = postgresql://user:pass@localhost/db
sqlalchemy.pool_size = 50
sqlalchemy.max_overflow = 100
sqlalchemy.pool_pre_ping = true
timezone = Asia/Jakarta
localization = id_ID.UTF-8
mail.host = localhost
mail.port = 25
mail.username = user@example.com
mail.password = FIXME
mail.sender_name = Example Name
retry.attempts = 3
# By default, the toolbar only appears for clients from IP addresses
# '127.0.0.1' and '::1'.
# debugtoolbar.hosts = 127.0.0.1 ::1
[alembic]
# path to migration scripts
script_location = iso8583_web/alembic
file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s
# file_template = %%(rev)s_%%(slug)s
###
# wsgi server configuration
###
[server:main]
use = egg:waitress#main
listen = localhost:6543
###
# logging configuration
# https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###
......@@ -41,18 +92,6 @@ formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)s %(message)s
[web]
port = 7000
threads = 12
[web_teller]
route_path = /rpc
host = kota_bogor
module = iso8583_web.scripts.views.jsonrpc
allowed_ip =
127.0.0.1
10.8.20.1
[host_agratek]
ip = 127.0.0.1
port = 10001
......@@ -92,6 +131,7 @@ streamer = bjb_with_suffix
module = opensipkd.iso8583.bjb.pbb.test
[module_opensipkd.iso8583.bjb.pbb.test]
db_url = postgresql://user:pass@localhost/db
request_bits =
18:6025
32:110
......
iso8583-pemda.ini
......@@ -40,6 +40,7 @@ format = %(asctime)s %(levelname)s %(message)s
db_url = postgresql://user:pass@localhost/db
db_pool_size = 50
db_max_overflow = 100
db_pool_pre_ping = true
persen_denda = 2
nip_pencatat = 999999999
......
iso8583_web/__init__.py
import os
import csv
import deform
from threading import Thread
from pkg_resources import resource_filename
from pyramid.i18n import get_localizer
from pyramid.threadlocal import get_current_request
......@@ -9,30 +10,16 @@ from pyramid_beaker import session_factory_from_settings
from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy
from pyramid_mailer import mailer_factory_from_settings
from sqlalchemy import engine_from_config
from .models import (
DBSession,
Base,
)
from .security import (
group_finder,
get_user,
)
from .tools.this_framework import get_locale_name
from .views import RemoveSlashNotFoundViewFactory
here = os.path.abspath(os.path.dirname(__file__))
routes_file = os.path.join(here, 'routes.csv')
def set_paths(config):
with open(routes_file) as f:
c = csv.DictReader(f)
for row in c:
path = row['path'] or '/' + row['name']
config.add_route(row['name'], path)
config.scan()
from .iso8583 import (
read_conf,
main_loop as iso8583_main_loop,
)
def translator(term):
......@@ -44,37 +31,42 @@ zpt_renderer = deform.ZPTRendererFactory(
[deform_template_dir], translator=translator)
deform.Form.set_default_renderer(zpt_renderer)
iso8583_threads = []
def start_iso8583(conf_file):
if not read_conf(conf_file):
return
thread = Thread(target=iso8583_main_loop, args=(conf_file,))
thread.daemon = True
iso8583_threads.append(thread)
thread.start()
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application.
"""
engine = engine_from_config(settings, 'sqlalchemy.')
DBSession.configure(bind=engine)
Base.metadata.bind = engine
session_factory = session_factory_from_settings(settings)
config = Configurator(
settings=settings,
root_factory='iso8583_web.models.ziggurat.RootFactory',
session_factory=session_factory,
locale_negotiator=get_locale_name)
config.include('.models')
config.include('pyramid_tm')
config.include('pyramid_beaker')
config.include('pyramid_chameleon')
config.include('pyramid_rpc.jsonrpc')
config.include('.renderers')
config.include('.routes')
authn_policy = AuthTktAuthenticationPolicy(
'sosecret', callback=group_finder, hashalg='sha512')
config.set_authentication_policy(authn_policy)
authz_policy = ACLAuthorizationPolicy()
config.set_authorization_policy(authz_policy)
config.add_request_method(get_user, 'user', reify=True)
config.add_notfound_view(RemoveSlashNotFoundViewFactory())
config.add_static_view('static', 'static', cache_max_age=3600)
config.add_static_view('deform_static', 'deform:static')
config.add_notfound_view(RemoveSlashNotFoundViewFactory())
config.add_translation_dirs('locale')
config.registry['mailer'] = mailer_factory_from_settings(settings)
set_paths(config)
config.scan()
start_iso8583(global_config['__file__'])
return config.make_wsgi_app()
alembic/env.py → iso8583_web/alembic/env.py
from __future__ import with_statement
"""Pyramid bootstrap environment. """
from alembic import context
from sqlalchemy import engine_from_config, pool
from logging.config import fileConfig
from pyramid.paster import get_appsettings, setup_logging
from sqlalchemy import engine_from_config
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
from iso8583_web.models.meta import Base
# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
config = context.config
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = None
setup_logging(config.config_file_name)
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
settings = get_appsettings(config.config_file_name)
target_metadata = Base.metadata
def run_migrations_offline():
......@@ -35,10 +25,7 @@ def run_migrations_offline():
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url, target_metadata=target_metadata, literal_binds=True)
context.configure(url=settings['sqlalchemy.url'])
with context.begin_transaction():
context.run_migrations()
......@@ -50,19 +37,19 @@ def run_migrations_online():
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix='sqlalchemy.',
poolclass=pool.NullPool)
engine = engine_from_config(settings, prefix='sqlalchemy.')
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata
)
connection = engine.connect()
context.configure(
connection=connection,
target_metadata=target_metadata
)
try:
with context.begin_transaction():
context.run_migrations()
finally:
connection.close()
if context.is_offline_mode():
......
iso8583_web/common.py 0 → 100644
import logging
from time import (
time,
sleep,
)
from sqlalchemy import exc
from opensipkd.tcp.connection import join_ip_port
from .iso8583.read_conf import name_conf
from .iso8583.connection import conn_mgr
log = logging.getLogger(__name__)
# Daftar job dari web request, berisi iso request
# key: ip:port, value: list of iso
web_request = {}
# Daftar job yang sedang diproses, yaitu menunggu iso response
# key: ip:port, value: list of stan (bit 11)
web_process = {}
# Daftar job yang sudah selesai, berisi iso response
# key: ip:port, value: dict of (key: stan, value: iso)
web_response = {}
# Nilai timeout dalam detik saat menunggu ISO-8583 response
ISO_TIMEOUT = 30
def append_web_process(ip_port, iso):
stan = iso.get_stan()
if ip_port in web_process:
web_process[ip_port].append(stan)
else:
web_process[ip_port] = [stan]
def append_web_response(ip_port, stan, iso):
if ip_port in web_response:
web_response[ip_port][stan] = iso
else:
web_response[ip_port] = {stan: iso}
# Abstract class. Inherit, please.
class BaseView:
def __init__(self, request):
self.request = request
def get_name(self): # Override, please
return 'unknown'
def get_allowed_ip(self): # Override, please
return ['0.0.0.0']
def validate(self):
allowed_ip = self.get_allowed_ip()
if '0.0.0.0' in allowed_ip:
return
if self.request.client_addr not in allowed_ip:
raise self.not_found_error(self.request.client_addr)
def log_prefix(self):
ip = self.request.client_addr
name = self.get_name()
mem_id = id(self.request)
return f'{ip} {name} {mem_id}'
def log_receive(self, msg):
prefix = self.log_prefix()
path = self.request.path
msg = f'{prefix} Receive {path} {msg}'
log.info(msg)
def log_send(self, msg):
prefix = self.log_prefix()
msg = f'{prefix} Send {msg}'
log.info(msg)
def log_debug(self, msg):
prefix = self.log_prefix()
msg = f'{prefix} {msg}'
log.debug(msg)
# Abstract class. Inherit, please.
class BaseIsoView(BaseView):
# Get ISO8583 connection
def get_connection(self):
conf = self.get_iso_conf()
found_conn = None
for ip_port, conn in conn_mgr:
ip, port = ip_port.split(':')
if conf['ip'] != ip:
continue
port = int(port)
if conf['port'] != port:
continue
found_conn = conn
if not found_conn:
raise self.not_found_error(conf['name'])
if not found_conn.running:
raise self.not_running_error(conf['name'])
return found_conn
def send_iso(self, conn, iso):
ip_port = join_ip_port(conn.conf['ip'], conn.conf['port'])
if ip_port in web_request:
web_request[ip_port].append(iso)
else:
web_request[ip_port] = [iso]
stan = iso.get_stan()
awal = time()
while True:
sleep(0.001)
if time() - awal > ISO_TIMEOUT:
raise self.timeout_error()
if ip_port not in web_response:
continue
result = web_response[ip_port]
if stan in result:
iso = result[stan]
del result[stan]
return iso
def get_iso_conf_name(self): # Override, please
return 'bjb'
def get_iso_conf(self):
name = self.get_iso_conf_name()
return name_conf[name]
def not_found_error(self, hostname):
msg = f'Host {hostname} tidak ditemukan di konfigurasi'
return Exception(msg)
def not_running_error(self, hostname):
msg = f'Host {hostname} belum terhubung'
return Exception(msg)
def timeout_error(self):
return Exception('Timeout')
SQL_CHECK = dict(
postgresql='SELECT 1',
oracle='SELECT 1 FROM dual')
def db_check(engine):
sql = SQL_CHECK[engine.dialect.name]
try:
engine.execute(sql)
except exc.DBAPIError as e:
if e.connection_invalidated:
log.debug('Database connection problem')
iso8583_web/scripts/forwarder.py → iso8583_web/iso8583/__init__.py
import os
import sys
import signal
import logging
from time import (
sleep,
time,
)
from threading import Thread
from waitress.server import create_server
from pyramid.config import Configurator
from pyramid.paster import setup_logging
from opensipkd.tcp.connection import join_ip_port
from opensipkd.string import exception_message
......@@ -16,11 +16,15 @@ from opensipkd.tcp.server import (
RequestHandler as BaseRequestHandler,
)
from opensipkd.tcp.client import Client as BaseClient
from ..read_conf import (
from ..common import (
web_request,
web_process,
append_web_process,
append_web_response,
)
from .read_conf import (
read_conf,
ip_conf,
web as web_conf,
web_path_conf,
listen_ports,
allowed_ips,
get_conf,
......@@ -28,21 +32,9 @@ from ..read_conf import (
)
from .tools import iso_to_dict
from .connection import conn_mgr
from .views import (
web_request,
web_process,
append_web_process,
append_web_response,
)
from .logger import (
get_log,
set_log,
log_info,
log_error,
log_debug,
log_web_info,
log_web_debug,
)
log = logging.getLogger(__name__)
# Jika kosong berarti perintah untuk mengakhiri aplikasi ini
......@@ -57,9 +49,6 @@ clients = {}
# Daftar penerjemah dokumen ISO8583
parser_threads = []
# Web server profile
web_server = {}
# Daftar antrian saat sebagai ISO8583 forwarder
# key: ip:port
# value: dict of (
......@@ -89,21 +78,21 @@ def to_str(s):
class Log:
def log_message(self, msg):
return '{ip} {name} {mem_id} {msg}'.format(
return 'ISO8583 {ip} {name} {mem_id} {msg}'.format(
ip=self.conf['ip'], name=self.conf['name'], mem_id=id(self),
msg=msg)
def log_info(self, msg):
msg = self.log_message(msg)
log_info(msg)
log.info(msg)
def log_error(self, msg):
msg = self.log_message(msg)
log_error(msg)
log.error(msg)
def log_debug(self, msg):
msg = self.log_message(msg)
log_debug(msg)
log.debug(msg)
def log_unknown(self):
msg = exception_message()
......@@ -166,7 +155,6 @@ class Server(BaseServer):
# Override
def verify_request(self, request, client_address):
client_ip = client_address[0]
log = get_log()
if client_ip in allowed_ips:
log.info('{} allowed'.format(client_ip))
return True
......@@ -228,7 +216,7 @@ class RequestHandler(BaseRequestHandler, CommonConnection):
def start_servers():
for listen_port in listen_ports:
listen_address = ('0.0.0.0', listen_port)
log_info('ISO8583 server listen at {}:{}'.format(*listen_address))
log.info('ISO8583 server listen at {}:{}'.format(*listen_address))
server = Server(listen_address, RequestHandler)
thread = create_thread(server.serve_forever)
servers[listen_port] = (server, thread)
......@@ -238,11 +226,11 @@ def start_servers():
def stop_servers(reason):
for listen_port in listen_ports:
server, thread = servers[listen_port]
log_debug('ISO8583 server shutdown')
log.debug('ISO8583 server shutdown')
server.shutdown()
log_debug('ISO8583 server thread join')
log.debug('ISO8583 server thread join')
thread.join()
log_debug('ISO8583 server thread joined')
log.debug('ISO8583 server thread joined')
sleep(1)
......@@ -307,11 +295,9 @@ def stop_connections(reason):
sleep(1)
for ip_port in clients:
client, thread = clients[ip_port]
msg = 'Thread {} join'.format(ip_port)
log_debug(msg)
log.debug(f'Thread {ip_port} join')
thread.join()
msg = 'Thread {} joined'.format(ip_port)
log_debug(msg)
log.debug(f'Thread {ip_port} joined')
#######################
......@@ -328,7 +314,7 @@ class Parser(Log):
# Override
def log_message(self, msg):
return '{ip} {name} {conn_id} -> {parser_id} {msg}'.format(
return 'ISO8583 {ip} {name} {conn_id} -> {parser_id} {msg}'.format(
ip=self.conf['ip'], name=self.conf['name'], conn_id=self.conn_id,
parser_id=self.parser_id, msg=msg)
......@@ -428,48 +414,6 @@ def get_connection_from_ip_port(ip_port):
return conn
#######
# Web #
#######
def start_web_server():
port = web_conf.get('port')
if not port:
return
thread_count = web_conf.get('threads')
host = '0.0.0.0'
with Configurator() as config:
config.include('pyramid_tm')
for path, cfg in web_path_conf.items():
cfg['module_obj'].pyramid_init(config)
config.scan(cfg['module'])
config.scan(__name__)
app = config.make_wsgi_app()
web_server['listener'] = server = create_server(
app, host=host, port=port, threads=thread_count)
web_server['thread'] = create_thread(server.run)
web_server['thread'].start()
log_web_info('listen at {}:{} with {} workers'.format(
host, port, thread_count))
def stop_web_server(reason):
if 'listener' not in web_server:
return
msg = 'stop because {}'.format(reason)
log_web_info(msg)
srv = web_server['listener']
srv.close()
while srv._map:
triggers = list(srv._map.values())
for trigger in triggers:
trigger.handle_close()
srv.maintenance(0)
srv.task_dispatcher.shutdown()
log_web_debug('thread join')
web_server['thread'].join()
log_web_debug('thread joined')
MSG_KILL_BY_SIGNAL = 'kill by signal {}'
MSG_KILL_BY_KEYBOARD = 'kill by keyboard interrupt'
......@@ -483,14 +427,6 @@ def out(sig=None, func=None):
reason = MSG_KILL_BY_KEYBOARD
stop_servers(reason)
stop_connections(reason)
stop_web_server(reason)
def usage(argv):
cmd = os.path.basename(argv[0])
print('usage: %s <config_uri>\n'
'(example: "%s test.ini")' % (cmd, cmd))
sys.exit(1)
def check_connection():
......@@ -560,18 +496,10 @@ def check_parser():
i -= 1
def main(argv=sys.argv):
if len(argv) != 2:
usage(argv)
config_uri = argv[1]
setup_logging(config_uri)
read_conf(config_uri)
set_log(config_uri, __file__)
def main_loop(conf_file):
setup_logging(conf_file)
running.append(True)
start_web_server()
start_servers()
# Antisipasi kill
signal.signal(signal.SIGTERM, out)
try:
while running:
check_connection()
......@@ -580,3 +508,10 @@ def main(argv=sys.argv):
sleep(0.5)
except KeyboardInterrupt:
out()
def main(argv=sys.argv):
conf_file = argv[1]
if read_conf(conf_file):
signal.signal(signal.SIGTERM, out)
main_loop(conf_file)
iso8583_web/read_conf.py → iso8583_web/iso8583/read_conf.py
......@@ -5,13 +5,12 @@ from configparser import (
)
listen_ports = list()
ip_conf = dict()
name_conf = dict()
allowed_ips = list()
PREFIX_SECTION = 'host_'
web = dict()
web_path_conf = dict()
listen_ports = list()
ip_conf = dict()
name_conf = dict()
allowed_ips = list()
def get_conf(ip, port):
......@@ -84,6 +83,7 @@ def append_others(cfg, conf, section):
def load_module(cfg, conf, section):
cfg['module'] = conf.get(section, 'module')
cfg['module_obj'] = get_module_object(cfg['module'])
append_others(cfg, conf, section)
module_section = 'module_' + cfg['module']
if conf.has_section(module_section):
append_others(cfg, conf, module_section)
......@@ -94,37 +94,12 @@ def load_module(cfg, conf, section):
pass
def read_web_conf(conf, section):
if section == 'web':
web['port'] = conf.getint(section, 'port')
web['threads'] = conf.getint(section, 'threads')
return
if section.find('web_') != 0:
return
cfg = dict()
cfg['name'] = section[4:]
cfg['timeout'] = get_int(conf, section, 'timeout', 30)
cfg['allowed_ip'] = get_list(conf, section, 'allowed_ip', [])
append_others(cfg, conf, section)
cfg['module_obj'] = get_module_object(cfg['module'])
try:
f_init = getattr(cfg['module_obj'], 'init')
f_init(cfg)
except AttributeError:
pass
web_path_conf[cfg['route_path']] = dict(cfg)
def read_host_conf(conf, section):
if section.find('host_') != 0:
return
if not get_boolean(conf, section, 'active', True):
return
def read_section(conf, section):
cfg = dict()
cfg['ip'] = conf.get(section, 'ip')
cfg['port'] = conf.getint(section, 'port')
ip_port = validate_ip_port(cfg)
cfg['name'] = section[5:]
cfg['name'] = section.lstrip(PREFIX_SECTION)
cfg['streamer'] = get_str(conf, section, 'streamer', 'none')
cfg['streamer_cls'] = get_streamer_class(cfg['streamer'])
cfg['ip'] = conf.get(section, 'ip')
......@@ -132,7 +107,6 @@ def read_host_conf(conf, section):
cfg['listen'] = get_boolean(conf, section, 'listen', True)
cfg['echo'] = get_boolean(conf, section, 'echo', not cfg['listen'])
cfg['timeout'] = get_int(conf, section, 'timeout', 60)
append_others(cfg, conf, section)
load_module(cfg, conf, section)
if cfg['listen']:
if cfg['port'] not in listen_ports:
......@@ -144,8 +118,11 @@ def read_host_conf(conf, section):
def read_conf(conf_file):
conf = ConfigParser()
conf['DEFAULT'] = dict(threads=12)
conf.read(conf_file)
for section in conf.sections():
read_web_conf(conf, section)
read_host_conf(conf, section)
if section.find(PREFIX_SECTION) != 0:
continue
if not get_boolean(conf, section, 'active', True):
continue
read_section(conf, section)
return ip_conf
iso8583_web/models/__init__.py
from sqlalchemy import (
Column,
Integer,
DateTime,
String,
Boolean,
)
from sqlalchemy.orm import (
sessionmaker,
scoped_session,
)
from sqlalchemy.ext.declarative import declarative_base
from zope.sqlalchemy import register
from ..tools.waktu import create_now
from sqlalchemy import engine_from_config
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import configure_mappers
import zope.sqlalchemy
# run configure_mappers after defining all of the models to ensure
# all relationships can be setup
configure_mappers()
def get_engine(settings, prefix='sqlalchemy.'):
return engine_from_config(settings, prefix)
session_factory = sessionmaker()
DBSession = scoped_session(session_factory)
register(DBSession)
Base = declarative_base()
def get_session_factory(engine):
factory = sessionmaker()
factory.configure(bind=engine)
return factory
class CommonModel(object):
def to_dict(self):
values = {}
for column in self.__table__.columns:
values[column.name] = getattr(self, column.name)
return values
def get_tm_session(session_factory, transaction_manager):
"""
Get a ``sqlalchemy.orm.Session`` instance backed by a transaction.
def to_dict_without_none(self):
values = {}
for column in self.__table__.columns:
val = getattr(self, column.name)
if val is not None:
values[column.name] = val
return values
This function will hook the session to the transaction manager which
will take care of committing any changes.
- When using pyramid_tm it will automatically be committed or aborted
depending on whether an exception is raised.
class BaseModel(CommonModel):
id = Column(Integer, nullable=False, primary_key=True)
- When using scripts you should wrap the session in a manager yourself.
For example::
import transaction
class LogModel(BaseModel):
created = Column(
DateTime(timezone=True), nullable=False, default=create_now)
engine = get_engine(settings)
session_factory = get_session_factory(engine)
with transaction.manager:
dbsession = get_tm_session(session_factory, transaction.manager)
"""
dbsession = session_factory()
zope.sqlalchemy.register(
dbsession, transaction_manager=transaction_manager)
return dbsession
class IsoModel(LogModel):
# Nama bank (bjb, btn) atau forwarder (mitracomm)
forwarder = Column(String(16), nullable=False)
ip = Column(String(15), nullable=False)
mti = Column(String(4), nullable=False)
is_send = Column(Boolean, nullable=False, default=True)
class HistoryModel(LogModel):
updated = Column(
DateTime(timezone=True), nullable=False, default=create_now)
def includeme(config):
"""
Initialize the model for a Pyramid app.
def save(self):
self.updated = create_now()
if not self.id:
self.created = self.updated
Activate this setup using ``config.include('linkaja_sambat.models')``.
"""
settings = config.get_settings()
settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'
# use pyramid_tm to hook the transaction lifecycle to the request
config.include('pyramid_tm')
# use pyramid_retry to retry a request when transient exceptions occur
config.include('pyramid_retry')
session_factory = get_session_factory(get_engine(settings))
config.registry['dbsession_factory'] = session_factory
# make request.dbsession available for use in Pyramid
config.add_request_method(
# r.tm is the transaction manager used by pyramid_tm
lambda r: get_tm_session(session_factory, r.tm),
'dbsession',
reify=True
)
iso8583_web/models/conf.py 0 → 100644
import json
from sqlalchemy import (
Column,
Integer,
String,
Text,
)
from .meta import Base
class Conf(Base):
__tablename__ = 'conf'
id = Column(Integer, primary_key=True)
nama = Column(String(100), unique=True)
nilai = Column(Text)
keterangan = Column(Text)
def as_boolean(self):
return self.nilai == 'true'
def as_int(self):
return int(self.nilai)
def as_float(self):
return float(self.nilai)
def as_list(self):
return self.nilai.split()
def as_dict(self):
return json.loads(self.nilai)
iso8583_web/models/log.py deleted 100644 → 0
from sqlalchemy import (
Column,
String,
Date,
DateTime,
BigInteger,
Integer,
Text,
Float,
Boolean,
ForeignKey,
UniqueConstraint,
)
from . import (
Base,
BaseModel,
LogModel,
IsoModel,
HistoryModel,
)
# h2h pbb / h2h bphtb / h2h padl / h2h webr
class Jenis(BaseModel, Base):
__tablename__ = 'log_jenis'
nama = Column(String(16), nullable=False, unique=True)
# INFO / ERROR / WARNING
class Kategori(BaseModel, Base):
__tablename__ = 'log_kategori'
nama = Column(String(7), nullable=False, unique=True)
class Log(LogModel, Base):
__tablename__ = 'log'
jenis_id = Column(Integer, ForeignKey(Jenis.id), nullable=False)
line = Column(Text, nullable=False)
# line_id berisi md5 dari line
line_id = Column(String(32), nullable=False)
tgl = Column(DateTime(timezone=True), nullable=False)
kategori_id = Column(
Integer, ForeignKey(Kategori.id), nullable=False)
__table_args__ = (UniqueConstraint('jenis_id', 'line_id'),)
class Iso(IsoModel, Base):
__tablename__ = 'log_iso'
id = Column(Integer, ForeignKey(Log.id), primary_key=True)
tgl = Column(DateTime(timezone=True), nullable=False)
jenis_id = Column(Integer, ForeignKey(Jenis.id), nullable=False)
kategori_id = Column(
Integer, ForeignKey(Kategori.id), nullable=False)
bit_002 = Column(Text)
bit_003 = Column(Text)
bit_004 = Column(Text)
bit_005 = Column(Text)
bit_006 = Column(Text)
bit_007 = Column(Text)
bit_008 = Column(Text)
bit_009 = Column(Text)
bit_010 = Column(Text)
bit_011 = Column(Text)
bit_012 = Column(Text)
bit_013 = Column(Text)
bit_014 = Column(Text)
bit_015 = Column(Text)
bit_016 = Column(Text)
bit_017 = Column(Text)
bit_018 = Column(Text)
bit_019 = Column(Text)
bit_020 = Column(Text)
bit_021 = Column(Text)
bit_022 = Column(Text)
bit_023 = Column(Text)
bit_024 = Column(Text)
bit_025 = Column(Text)
bit_026 = Column(Text)
bit_027 = Column(Text)
bit_028 = Column(Text)
bit_029 = Column(Text)
bit_030 = Column(Text)
bit_031 = Column(Text)
bit_032 = Column(Text)
bit_033 = Column(Text)
bit_034 = Column(Text)
bit_035 = Column(Text)
bit_036 = Column(Text)
bit_037 = Column(Text)
bit_038 = Column(Text)
bit_039 = Column(Text)
bit_040 = Column(Text)
bit_041 = Column(Text)
bit_042 = Column(Text)
bit_043 = Column(Text)
bit_044 = Column(Text)
bit_045 = Column(Text)
bit_046 = Column(Text)
bit_047 = Column(Text)
bit_048 = Column(Text)
bit_049 = Column(Text)
bit_050 = Column(Text)
bit_051 = Column(Text)
bit_052 = Column(Text)
bit_053 = Column(Text)
bit_054 = Column(Text)
bit_055 = Column(Text)
bit_056 = Column(Text)
bit_057 = Column(Text)
bit_058 = Column(Text)
bit_059 = Column(Text)
bit_060 = Column(Text)
bit_061 = Column(Text)
bit_062 = Column(Text)
bit_063 = Column(Text)
bit_064 = Column(Text)
bit_065 = Column(Text)
bit_066 = Column(Text)
bit_067 = Column(Text)
bit_068 = Column(Text)
bit_069 = Column(Text)
bit_070 = Column(Text)
bit_071 = Column(Text)
bit_072 = Column(Text)
bit_073 = Column(Text)
bit_074 = Column(Text)
bit_075 = Column(Text)
bit_076 = Column(Text)
bit_077 = Column(Text)
bit_078 = Column(Text)
bit_079 = Column(Text)
bit_080 = Column(Text)
bit_081 = Column(Text)
bit_082 = Column(Text)
bit_083 = Column(Text)
bit_084 = Column(Text)
bit_085 = Column(Text)
bit_086 = Column(Text)
bit_087 = Column(Text)
bit_088 = Column(Text)
bit_089 = Column(Text)
bit_090 = Column(Text)
bit_091 = Column(Text)
bit_092 = Column(Text)
bit_093 = Column(Text)
bit_094 = Column(Text)
bit_095 = Column(Text)
bit_096 = Column(Text)
bit_097 = Column(Text)
bit_098 = Column(Text)
bit_099 = Column(Text)
bit_100 = Column(Text)
bit_101 = Column(Text)
bit_102 = Column(Text)
bit_103 = Column(Text)
bit_104 = Column(Text)
bit_105 = Column(Text)
bit_106 = Column(Text)
bit_107 = Column(Text)
bit_108 = Column(Text)
bit_109 = Column(Text)
bit_110 = Column(Text)
bit_111 = Column(Text)
bit_112 = Column(Text)
bit_113 = Column(Text)
bit_114 = Column(Text)
bit_115 = Column(Text)
bit_116 = Column(Text)
bit_117 = Column(Text)
bit_118 = Column(Text)
bit_119 = Column(Text)
bit_120 = Column(Text)
bit_121 = Column(Text)
bit_122 = Column(Text)
bit_123 = Column(Text)
bit_124 = Column(Text)
bit_125 = Column(Text)
bit_126 = Column(Text)
bit_127 = Column(Text)
bit_128 = Column(Text)
class Conf(HistoryModel, Base):
__tablename__ = 'log_conf'
nama = Column(String(64), nullable=False, unique=True)
nilai = Column(Text)
nilai_int = Column(Integer)
nilai_float = Column(Float)
nilai_bool = Column(Boolean)
class Method(BaseModel, Base):
__tablename__ = 'iso_method'
nama = Column(String(16), nullable=False, unique=True)
class Bank(BaseModel, Base):
__tablename__ = 'bank'
nama = Column(String(32), nullable=False, unique=True)
class Summary(BaseModel, Base):
__tablename__ = 'iso_summary'
__table_args__ = (
UniqueConstraint('jenis_id', 'tgl', 'method_id', 'bank_id'),
)
jenis_id = Column(Integer, ForeignKey(Jenis.id), nullable=False)
tgl = Column(Date, nullable=False)
method_id = Column(Integer, ForeignKey(Method.id), nullable=False)
bank_id = Column(Integer, ForeignKey(Bank.id), nullable=False)
trx_count = Column(Integer, nullable=False, default=0)
trx_amount = Column(Float, nullable=False, default=0)
iso8583_web/models/meta.py 0 → 100644
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.schema import MetaData
# Recommended naming convention used by Alembic, as various different database
# providers will autogenerate vastly different names making migrations more
# difficult. See: http://alembic.zzzcomputing.com/en/latest/naming.html
NAMING_CONVENTION = {
"ix": "ix_%(column_0_label)s",
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(constraint_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
}
metadata = MetaData(naming_convention=NAMING_CONVENTION)
Base = declarative_base(metadata=metadata)
class CommonModel(object):
def to_dict(self):
values = {}
for column in self.__table__.columns:
values[column.name] = getattr(self, column.name)
return values
def to_dict_without_none(self):
values = {}
for column in self.__table__.columns:
val = getattr(self, column.name)
if val is not None:
values[column.name] = val
return values
iso8583_web/models/ziggurat.py
......@@ -9,31 +9,30 @@ from ziggurat_foundations.models.base import BaseModel
from ziggurat_foundations.models.external_identity import ExternalIdentityMixin
from ziggurat_foundations.models.group import GroupMixin
from ziggurat_foundations.models.group_permission import GroupPermissionMixin
from ziggurat_foundations.models.group_resource_permission import GroupResourcePermissionMixin
from ziggurat_foundations.models.group_resource_permission \
import GroupResourcePermissionMixin
from ziggurat_foundations.models.resource import ResourceMixin
from ziggurat_foundations.models.user import UserMixin
from ziggurat_foundations.models.user_group import UserGroupMixin
from ziggurat_foundations.models.user_permission import UserPermissionMixin
from ziggurat_foundations.models.user_resource_permission import UserResourcePermissionMixin
from ziggurat_foundations.models.user_resource_permission \
import UserResourcePermissionMixin
from ziggurat_foundations import ziggurat_model_init
from . import (
from .meta import (
Base,
DBSession,
CommonModel,
)
# this is needed for scoped session approach like in pylons 1.0
ziggurat_foundations.models.DBSession = DBSession
# optional for folks who pass request.db to model methods
# Base is sqlalchemy's Base = declarative_base() from your project
class Group(GroupMixin, Base, CommonModel):
pass
class GroupPermission(GroupPermissionMixin, Base):
pass
class UserGroup(UserGroupMixin, Base):
pass
......@@ -41,9 +40,10 @@ class UserGroup(UserGroupMixin, Base):
class GroupResourcePermission(GroupResourcePermissionMixin, Base):
__table_args__ = (
PrimaryKeyConstraint(
"group_id",
"resource_id",
"perm_name"),)
'group_id',
'resource_id',
'perm_name'),)
class Resource(ResourceMixin, Base):
# ... your own properties....
......@@ -57,41 +57,45 @@ class Resource(ResourceMixin, Base):
acls.extend([(Allow, self.owner_user_id, ALL_PERMISSIONS,), ])
if self.owner_group_id:
acls.extend([(Allow, "group:%s" % self.owner_group_id,
acls.extend([(Allow, 'group:%s' % self.owner_group_id,
ALL_PERMISSIONS,), ])
return acls
class UserPermission(UserPermissionMixin, Base):
pass
class UserResourcePermission(UserResourcePermissionMixin, Base):
pass
class User(UserMixin, Base, CommonModel):
# ... your own properties....
pass
class ExternalIdentity(ExternalIdentityMixin, Base):
pass
# you can define multiple resource derived models to build a complex
# application like CMS, forum or other permission based solution
#class Entry(Resource):
# """
# Resource of `entry` type
# """
# class Entry(Resource):
# '''
# Resource of `entry` type
# '''
# __tablename__ = 'entries'
# __mapper_args__ = {'polymorphic_identity': 'entry'}
# __tablename__ = 'entries'
# __mapper_args__ = {'polymorphic_identity': 'entry'}
# resource_id = sa.Column(sa.Integer(),
# sa.ForeignKey('resources.resource_id',
# onupdate='CASCADE',
# ondelete='CASCADE', ),
# primary_key=True, )
# ... your own properties....
# some_property = sa.Column(sa.UnicodeText())
# resource_id = sa.Column(sa.Integer(),
# sa.ForeignKey('resources.resource_id',
# onupdate='CASCADE',
# ondelete='CASCADE', ),
# primary_key=True, )
# ... your own properties....
# some_property = sa.Column(sa.UnicodeText())
class RootFactory:
......@@ -100,12 +104,12 @@ class RootFactory:
(Allow, Authenticated, 'view'),
(Allow, 'group:1', ALL_PERMISSIONS),
]
for gp in DBSession.query(GroupPermission):
for gp in request.dbsession.query(GroupPermission):
acl_name = 'group:{}'.format(gp.group_id)
self.__acl__.append((Allow, acl_name, gp.perm_name))
#ziggurat_model_init(User, Group, UserGroup, GroupPermission, passwordmanager=None)
ziggurat_model_init(User, Group, UserGroup, GroupPermission, UserPermission,
UserResourcePermission, GroupResourcePermission, Resource,
ExternalIdentity, passwordmanager=None)
ziggurat_model_init(
User, Group, UserGroup, GroupPermission, UserPermission,
UserResourcePermission, GroupResourcePermission, Resource,
ExternalIdentity, passwordmanager=None)
iso8583_web/renderers.py
class CSVRenderer(object):
def __init__(self, info):
pass
from pyramid_linkaja.structure import RENDERER
def __call__(self, value, system):
''' value bertipe LinkAjaDataResponse '''
request = system.get('request')
if request is not None:
response = request.response
ct = response.content_type
if ct == response.default_content_type:
response.content_type = 'text/csv'
return str(value)
def includeme(config):
config.add_renderer(RENDERER, 'pyramid_linkaja.renderer.Renderer')
iso8583_web/routes.csv deleted 100644 → 0
name,path
home,/
login
logout
change-password
change-password-done
reset-password
reset-password-sent
login-by-code-failed
user
user-add,/user/add
user-edit,/user/{id}
user-delete,/user/{id}/delete
group
group-add,/group/add
group-edit,/group/{id}
group-delete,/group/{id}/delete
log
iso8583_web/routes.py 0 → 100644
def includeme(config):
config.add_static_view('static', 'static', cache_max_age=3600)
config.add_static_view('deform_static', 'deform:static')
config.add_route('home', '/')
config.add_route('login', '/login')
config.add_route('logout', '/logout')
config.add_route('change-password', '/change-password')
config.add_route('change-password-done', '/change-password-done')
config.add_route('reset-password', '/reset-password')
config.add_route('reset-password-sent', '/reset-password-sent')
config.add_route('login-by-code-failed', '/login-by-code-failed')
config.add_route('user', '/user')
config.add_route('user-add', '/user/add')
config.add_route('user-edit', '/user/{id}')
config.add_route('user-delete', '/user/{id}/delete')
config.add_route('group', '/group')
config.add_route('group-add', '/group/add')
config.add_route('group-edit', '/group/{id}')
config.add_route('group-delete', '/group/{id}/delete')
iso8583_web/scripts/data/bank.csv deleted 100644 → 0
id,nama
2,BRI
8,Mandiri
9,BNI
13,Permata
14,BCA
22,Niaga
110,BJB
iso8583_web/scripts/data/conf.csv 0 → 100644
nama,nilai,keterangan
linkaja allowed ip,127.0.0.1 10.8.50.22 10.14.0.203 10.11.0.118 10.11.0.35 202.43.164.162,list lebih dari satu pisahkan dengan spasi. 0.0.0.0 berarti boleh semua.
linkaja pbb prefix id 3271,PBB Kota Bogor
linkaja pbb prefix id 3676,PBB Kota Tangerang
linkaja pbb prefix id 3275,PBB Kota Bekasi
linkaja pbb to iso8583,bjb,Route ke section [host_bjb] di file konfigurasi
linkaja sambat aggregator code,W0227
linkaja sambat bank base url,http://localhost:8080/sam-core-app2/rest/SAMWebService
linkaja sambat bank name,bjb,Untuk field linkaja_bank.conf_name
linkaja sambat ca code,CA1864
rpc allowed ip,0.0.0.0
rpc to iso8583,kabupaten_cirebon_payment_point,Route ke section [host_kabupaten_cirebon_payment_point] di file konfigurasi
iso8583_web/scripts/data/iso_method.csv deleted 100644 → 0
id,nama
1,Inquiry
2,Payment
3,Reversal
iso8583_web/scripts/data/jenis.csv deleted 100644 → 0
id,nama
1,H2H PBB
2,H2H BPHTB
3,H2H PADL
4,H2H Web Register
5,H2H Perijinan
7,H2H T-SAMSAT
iso8583_web/scripts/data/kategori.csv deleted 100644 → 0
id,nama
1,Info
2,Error
3,Warning
iso8583_web/scripts/initialize_db.py
......@@ -2,9 +2,9 @@ import os
import sys
import csv
import subprocess
import transaction
from getpass import getpass
from sqlalchemy import engine_from_config
from argparse import ArgumentParser
import transaction
from ziggurat_foundations.models.services.user import UserService
from pyramid.paster import (
get_appsettings,
......@@ -16,8 +16,9 @@ from pyramid.i18n import (
Translations,
)
from ..models import (
DBSession,
Base,
get_engine,
get_session_factory,
get_tm_session,
)
from ..models.ziggurat import (
Group,
......@@ -25,12 +26,11 @@ from ..models.ziggurat import (
UserGroup,
User,
)
from ..models.log import (
Bank,
Method,
Jenis,
Kategori,
from ..models.meta import (
metadata,
Base,
)
from ..models.conf import Conf
domain = 'initialize_db'
......@@ -52,13 +52,6 @@ class MyLocalizer:
return self.localizer.translate(ts)
def usage(argv):
cmd = os.path.basename(argv[0])
print('usage: %s <config_uri>\n'
'(example: "%s development.ini")' % (cmd, cmd))
sys.exit(1)
def read_file(filename):
f = open(filename)
s = f.read()
......@@ -66,17 +59,12 @@ def read_file(filename):
return s
def alembic_run(ini_file, url):
def alembic_run(ini_file):
bin_path = os.path.split(sys.executable)[0]
alembic_bin = os.path.join(bin_path, 'alembic')
command = (alembic_bin, 'upgrade', 'head')
s = read_file(ini_file)
s = s.replace('{db_url}', url)
f = open('alembic.ini', 'w')
f.write(s)
f.close()
subprocess.call(command)
os.remove('alembic.ini')
alembic_bin = os.path.join(bin_path, 'alembic')
command = (alembic_bin, '-c', ini_file, 'upgrade', 'head')
if subprocess.call(command) != 0:
sys.exit()
def get_file(filename):
......@@ -87,7 +75,7 @@ def get_file(filename):
def ask_password(name):
localizer = MyLocalizer()
data = dict(name=name)
data = dict(name=name)
t_msg1 = _(
'ask-password-1', default='Enter new password for ${name}: ',
mapping=data)
......@@ -108,16 +96,17 @@ def ask_password(name):
def restore_csv(table, filename):
DBSession = my_registry['dbsession']
q = DBSession.query(table)
if q.first():
return
with get_file(filename) as f:
with get_file(filename) as f:
reader = csv.DictReader(f)
for cf in reader:
row = table()
for fieldname in cf:
val = cf[fieldname]
if not val:
if not val:
continue
setattr(row, fieldname, val)
DBSession.add(row)
......@@ -125,7 +114,8 @@ def restore_csv(table, filename):
def append_csv(table, filename, keys):
with get_file(filename) as f:
DBSession = my_registry['dbsession']
with get_file(filename) as f:
reader = csv.DictReader(f)
filter_ = dict()
for cf in reader:
......@@ -138,33 +128,43 @@ def append_csv(table, filename, keys):
row = table()
for fieldname in cf:
val = cf[fieldname]
if not val:
if not val:
continue
setattr(row, fieldname, val)
DBSession.add(row)
def setup_models(dbsession):
metadata.create_all(dbsession.bind)
append_csv(Conf, 'conf.csv', ['nama'])
if restore_csv(User, 'users.csv'):
dbsession.flush()
q = dbsession.query(User).filter_by(id=1)
user = q.first()
password = ask_password(user.user_name)
UserService.set_password(user, password)
append_csv(Group, 'groups.csv', ['group_name'])
restore_csv(UserGroup, 'users_groups.csv')
def parse_args(argv):
parser = ArgumentParser()
parser.add_argument(
'config_uri',
help='Configuration file, e.g., development.ini',
)
return parser.parse_args(argv[1:])
def main(argv=sys.argv):
if len(argv) != 2:
usage(argv)
config_uri = argv[1]
setup_logging(config_uri)
settings = get_appsettings(config_uri)
my_registry['settings'] = settings
engine = engine_from_config(settings, 'sqlalchemy.')
Base.metadata.bind = engine
Base.metadata.create_all()
alembic_run('alembic.ini.tpl', settings['sqlalchemy.url'])
args = parse_args(argv)
setup_logging(args.config_uri)
my_registry['settings'] = settings = get_appsettings(args.config_uri)
engine = get_engine(settings)
Base.metadata.create_all(engine)
alembic_run(args.config_uri)
session_factory = get_session_factory(engine)
with transaction.manager:
if restore_csv(User, 'users.csv'):
DBSession.flush()
q = DBSession.query(User).filter_by(id=1)
user = q.first()
password = ask_password(user.user_name)
UserService.set_password(user, password)
append_csv(Group, 'groups.csv', ['group_name'])
restore_csv(UserGroup, 'users_groups.csv')
restore_csv(Bank, 'bank.csv')
restore_csv(Method, 'iso_method.csv')
restore_csv(Jenis, 'jenis.csv')
restore_csv(Kategori, 'kategori.csv')
my_registry['dbsession'] = dbsession = get_tm_session(
session_factory, transaction.manager)
setup_models(dbsession)
iso8583_web/scripts/logger.py deleted 100644 → 0
import logging
from configparser import ConfigParser
logs = []
def set_log(conf_file, name=None):
if not name:
name = __file__
conf = ConfigParser()
conf.read(conf_file)
log = logging.getLogger(name)
log.setLevel(conf.get('logger_root', 'level'))
logs.append(log)
def get_log():
return logs[0]
def log_info(s):
log = get_log()
log.info(s)
def log_error(s):
log = get_log()
log.error(s)
def log_debug(s):
log = get_log()
log.debug(s)
def log_web_msg(s):
return 'Web server {}'.format(s)
def log_web_info(s):
msg = log_web_msg(s)
log = get_log()
log.info(msg)
def log_web_error(s):
msg = log_web_msg(s)
log = get_log()
log.error(msg)
def log_web_debug(s):
msg = log_web_msg(s)
log = get_log()
log.debug(msg)
iso8583_web/scripts/set_conf.py 0 → 100644
import sys
from argparse import ArgumentParser
import transaction
from pyramid.paster import get_appsettings
from ..models import (
get_engine,
get_session_factory,
get_tm_session,
)
from ..models.conf import Conf
def parse_args(argv):
parser = ArgumentParser()
parser.add_argument('conf', help='File konfigurasi')
parser.add_argument('--nama')
parser.add_argument('--nilai')
return parser.parse_args(argv)
def out(s):
print(s)
sys.exit()
def main(argv=sys.argv[1:]):
args = parse_args(argv)
settings = get_appsettings(args.conf)
engine = get_engine(settings)
session_factory = get_session_factory(engine)
with transaction.manager:
db_session = get_tm_session(session_factory, transaction.manager)
if not args.nama:
q = db_session.query(Conf).order_by(Conf.nama)
for row in q:
print(f'"{row.nama}" = "{row.nilai}"')
sys.exit()
q = db_session.query(Conf).filter_by(nama=args.nama)
row = q.first()
if not row:
out(f'Tidak ditemukan')
if not args.nilai:
out(f'"{args.nama}" = "{row.nilai}"')
if row.nilai == args.nilai:
out(f'Masih sama')
old = row.nilai
row.nilai = args.nilai
db_session.add(row)
print(f'Konfigurasi telah diubah')
print(f'dari "{old}"')
print(f'menjadi "{args.nilai}"')
iso8583_web/scripts/views/__init__.py deleted 100644 → 0
from time import (
time,
sleep,
)
from pyramid.view import notfound_view_config
from opensipkd.tcp.connection import join_ip_port
from iso8583_web.read_conf import (
name_conf,
web_path_conf,
)
from iso8583_web.scripts.logger import (
log_web_info,
log_web_error,
)
from ..connection import conn_mgr
from ..logger import log_web_info
# Daftar job dari web request, berisi iso request
# key: ip:port, value: list of iso
web_request = {}
# Daftar job yang sedang diproses, yaitu menunggu iso response
# key: ip:port, value: list of stan (bit 11)
web_process = {}
# Daftar job yang sudah selesai, berisi iso response
# key: ip:port, value: dict of (key: stan, value: iso)
web_response = {}
def append_web_process(ip_port, iso):
stan = iso.get_stan()
if ip_port in web_process:
web_process[ip_port].append(stan)
else:
web_process[ip_port] = [stan]
def append_web_response(ip_port, stan, iso):
if ip_port in web_response:
web_response[ip_port][stan] = iso
else:
web_response[ip_port] = {stan: iso}
def get_web_conf(request):
return web_path_conf.get(request.path)
def log_prefix(request):
web_conf = get_web_conf(request)
if web_conf:
name = web_conf['name']
else:
name = 'unknown'
return f'{request.client_addr} {name} {id(request)}'
def log_receive(request, msg, error=False):
prefix = log_prefix(request)
msg = f'{prefix} Receive {msg}'
if error:
log_web_error(msg)
else:
log_web_info(msg)
class WebJob:
def __init__(self, conn, iso, conf=dict()):
self.conn = conn
self.iso = iso
self.conf = conf
def get_response(self):
ip_port = join_ip_port(self.conn.conf['ip'], self.conn.conf['port'])
if ip_port in web_request:
web_request[ip_port].append(self.iso)
else:
web_request[ip_port] = [self.iso]
stan = self.iso.get_stan()
awal = time()
while True:
sleep(0.001)
if time() - awal > self.conf.get('timeout', 30):
raise self.timeout_error()
if ip_port not in web_response:
continue
result = web_response[ip_port]
if stan in result:
iso = result[stan]
del result[stan]
return iso
def timeout_error(self):
return Exception('Timeout')
class View:
def __init__(self, request):
self.request = request
def log_prefix(self):
return log_prefix(self.request)
def log_receive(self, msg, error=False):
msg = '{} {} {}'.format(self.log_prefix(), 'Receive', msg)
if error:
log_web_error(msg)
else:
log_web_info(msg)
def log_send(self, msg, error=False):
msg = '{} {} {}'.format(self.log_prefix(), 'Send', msg)
if error:
log_web_error(msg)
else:
log_web_info(msg)
# Get ISO8583 connection
def get_connection(self):
web_conf = self.get_web_conf()
if not web_conf:
raise self.not_found_error(self.request.client_addr)
name = web_conf['host']
conf = name_conf[name]
found_conn = None
for ip_port, conn in conn_mgr:
ip, port = ip_port.split(':')
if conf['ip'] != ip:
continue
port = int(port)
if conf['port'] != port:
continue
found_conn = conn
if not found_conn:
raise self.not_found_error(name)
if not found_conn.running:
raise self.not_running_error(name)
return found_conn
def get_web_conf(self):
return get_web_conf(self.request)
def get_iso_conf(self):
web_conf = self.get_web_conf()
name = web_conf['host']
return name_conf[name]
def validate(self):
conf = self.get_web_conf()
if not conf['allowed_ip']:
return
if self.request.client_addr not in conf['allowed_ip']:
raise self.not_found_error(self.request.client_addr)
def not_found_error(self, hostname):
msg = 'Host {} tidak ditemukan di konfigurasi'.format(hostname)
return Exception(msg)
def not_running_error(self, hostname):
msg = 'Host {} belum terhubung'.format(hostname)
return Exception(msg)
def get_web_job_cls(self):
return WebJob
def web_job(self, conn, iso):
conf = self.get_web_conf()
cls = self.get_web_job_cls()
job = cls(conn, iso, conf)
return job.get_response()
@notfound_view_config()
def view_not_found(request):
msg = f'Path {request.path} tidak ada'
log_receive(request, msg, True)
return request.exception
iso8583_web/scripts/views/jsonrpc.py deleted 100644 → 0
from pyramid_rpc.jsonrpc import jsonrpc_method
from opensipkd.jsonrpc.exc import (
JsonRpcInvalidParams,
JsonRpcBankNotFound,
JsonRpcBillerNetwork,
)
from ..tools import iso_to_dict
from ..logger import log_web_info
from . import (
WebJob as BaseWebJob,
View as BaseView,
)
ROUTE = 'rpc'
conf = dict()
class WebJob(BaseWebJob):
def timeout_error(self): # override
raise JsonRpcBillerNetwork(message='Timeout')
class View(BaseView):
def get_web_job_cls(self): # Override
return WebJob
def get_response(self, rpc_method, p=dict(), iso_method=None):
self.log_receive(rpc_method, p)
conn = self.get_connection()
iso_func = getattr(conn.job, iso_method or rpc_method)
iso_req = p and iso_func(p) or iso_func()
iso_resp = self.web_job(conn, iso_req)
data = iso_to_dict(iso_resp)
r = dict(code=0, message='OK', data=data)
self.log_send(r)
return r
def not_found_error(self, hostname): # Override
msg = 'Host {} tidak ditemukan di konfigurasi'.format(hostname)
return JsonRpcBankNotFound(message=msg)
def not_running_error(self, hostname): # Override
msg = 'Host {} belum terhubung'.format(hostname)
return JsonRpcBankNotFound(message=msg)
def log_receive(self, method, p):
msg = '{} {}'.format(method, p)
BaseView.log_receive(self, msg)
def log_send(self, p):
BaseView.log_send(self, p)
@jsonrpc_method(endpoint=ROUTE)
def echo(self, p):
self.validate()
return self.get_response('echo', 'echo_request')
@jsonrpc_method(endpoint=ROUTE)
def inquiry(self, p):
self.validate()
return self.get_response('inquiry', p)
@jsonrpc_method(endpoint=ROUTE)
def payment(self, p):
self.validate()
return self.get_response('payment', p)
@jsonrpc_method(endpoint=ROUTE)
def reversal(self, p):
self.validate()
return self.get_response('reversal', p)
# Dipanggil read_conf.py
def init(cfg):
conf.update(cfg)
# Dipanggil forwarder.py
def pyramid_init(config):
config.include('pyramid_rpc.jsonrpc')
config.add_jsonrpc_endpoint(ROUTE, conf['route_path'])
log_web_info('JSON-RPC route path {}'.format(conf['route_path']))
iso8583_web/scripts/views/linkaja/__init__.py deleted 100644 → 0
from pyramid.view import view_config
from deform import ValidationFailure
from sqlalchemy import create_engine
from sqlalchemy.orm import (
sessionmaker,
scoped_session,
)
from zope.sqlalchemy import register
from iso8583_web.read_conf import web_path_conf
from iso8583_web.scripts.logger import (
log_web_info,
log_web_error,
)
from opensipkd.iso8583.bjb.pbb.agratek.models import Rpc
from pyramid_linkaja.responses import (
InquiryResponse,
get_trx_date,
get_method,
is_inquiry,
is_payment,
)
from pyramid_linkaja.exceptions import (
BaseError,
NeedPostError,
InternalError,
BillRefNotFound,
)
from pyramid_linkaja.structure import RENDERER
from pyramid_linkaja.form import get_form
from .. import (
WebJob as BaseWebJob,
View as BaseView,
)
def get_web_conf(request):
return web_path_conf.get(request.path)
def log_prefix(request):
web_conf = get_web_conf(request)
name = web_conf['name']
return f'{request.client_addr} {name} {id(request)}'
def log_receive(self, msg, error=False):
msg = '{} {} {}'.format(self.log_prefix(), 'Receive', msg)
if error:
log_web_error(msg)
else:
log_web_info(msg)
def pyramid_init(pyramid_config, h2h_conf, route_name):
pyramid_config.add_renderer(RENDERER, 'pyramid_linkaja.renderer.Renderer')
path = h2h_conf['route_path']
pyramid_config.add_route(route_name, path)
log_web_info(f'LinkAja route path {path}')
pool_size = int(h2h_conf.get('db_pool_size', 50))
max_overflow = int(h2h_conf.get('db_max_overflow', 100))
engine = create_engine(
h2h_conf['db_url'], pool_size=pool_size, max_overflow=max_overflow)
session_factory = sessionmaker(bind=engine)
h2h_conf['db_session'] = scoped_session(session_factory)
register(h2h_conf['db_session'])
class WebJob(BaseWebJob):
def timeout_error(self): # override
return TimeoutError()
class View(BaseView):
def get_web_job_cls(self): # Override
return WebJob
def not_found_error(self, hostname): # Override
return HostError(hostname)
def not_running_error(self, hostname): # Override
msg = f'Host {hostname} belum terhubung'
return InternalError(msg, 'Sedang offline')
def get_web_conf(self):
return get_web_conf(self.request)
def get_inquiry(self, data):
DBSession = self.get_db_session()
bill_ref = int(data['bill_ref'])
q = DBSession.query(Rpc).filter_by(id=bill_ref)
return q.first()
def get_payment(self, data):
DBSession = self.get_db_session()
bill_ref = int(data['bill_ref'])
q = DBSession.query(Rpc).filter_by(
inquiry_id=bill_ref, trx_type='022')
q = q.order_by(Rpc.id.desc())
return q.first()
def create_db_log(self, data, inq, pay):
DBSession = self.get_db_session()
web_conf = self.get_web_conf()
row = Rpc(
ip=self.request.client_addr,
conf_name=web_conf['name'],
merchant=data['merchant'],
terminal=data['terminal'],
trx_type=data['trx_type'],
msisdn=data['msisdn'],
acc_no=data['acc_no'])
row.trx_date = get_trx_date(data),
if data.get('msg'):
row.msg = data['msg']
if data.get('amount'):
row.amount = int(data['amount'])
if not is_inquiry(data):
row.inquiry_id = inq and inq.id or pay.inquiry_id
row.ntb = data['trx_id']
return row
def view(self):
self.validate()
if not self.request.POST:
self.log_receive(f'GET {self.request.GET}')
raise NeedPostError()
items = self.request.POST.items()
self.log_receive(f'POST {dict(items)}')
items = self.request.POST.items()
form = get_form()
try:
c = form.validate(items)
except ValidationFailure as e:
d = e.error.asdict()
for field, err in d.items():
msg = f'{field} {err}'
break
raise InternalError(msg)
data = dict(c)
return self.get_response(data)
def get_db_session(self): # Override, please
pass
def get_invoice_id(self, data):
return data['acc_no']
def get_invoice_id_from_inquiry(self, inq):
return inq.acc_no
def prepare_response(self, data):
inq = pay = None
if is_inquiry(data):
invoice_id = self.get_invoice_id(data)
p = dict(invoice_id=invoice_id)
else:
p = dict(amount=data['amount'], ntb=data['trx_id'])
if is_payment(data):
inq = self.get_inquiry(data)
if not inq:
raise BillRefNotFound()
p['invoice_id'] = self.get_invoice_id_from_inquiry(inq)
else:
pay = self.get_payment(data)
if not pay:
raise BillRefNotFound()
inq = self.get_inquiry(data)
p['invoice_id'] = self.get_invoice_id_from_inquiry(inq)
p['stan'] = pay.stan
p['db_session'] = self.get_db_session()
return p, inq, pay
def get_response(self, data): # Override, please
pass
iso8583_web/scripts/views/linkaja/pbb.py deleted 100644 → 0
import transaction
from pyramid.view import view_config
from pyramid.response import Response
from ISO8583.ISOErrors import BitNotSet
from opensipkd.string import FixLength
from opensipkd.iso8583.bjb.pbb.structure import INVOICE_PROFILE
from opensipkd.iso8583.bjb.pbb.agratek.models import Log
from pyramid_linkaja.exceptions import (
InvoiceIdError,
TrxTypeError,
AlreadyPaidError,
BaseError,
AmountError,
BillRefNotFound,
PaymentNotFound,
LinkError,
BaseError,
)
from pyramid_linkaja.responses import (
InquiryResponse,
PaymentResponse,
get_method,
get_template_response,
is_inquiry,
is_payment,
is_reversal,
)
from pyramid_linkaja.decorator import csv_method
from iso8583_web.scripts.logger import log_web_error
from iso8583_web.scripts.tools import iso_to_dict
from .. import log_prefix
from . import View as BaseView
from . import pyramid_init as base_pyramid_init
ROUTE = 'linkaja-pbb'
conf = dict()
def get_db_session():
return conf['db_session']
def profile2name(profile):
msg = [profile['Nama'].strip()]
fields = [('Tagihan', 'Pok'), ('Denda', 'Den'), ('Discount', 'Disk')]
for field, label in fields:
s = profile[field].strip().lstrip('0')
if s:
msg.append(f'{label} {s}')
fields = ['Nama Kelurahan', 'Nama Kecamatan', 'Lokasi']
for field in fields:
s = profile[field].strip()
if s:
msg.append(s)
return ', '.join(msg)
def get_ok_notify(data):
for key in conf['notification_message']:
if data['acc_no'].find(key) == 0:
return conf['notification_message'][key]
return ''
@view_config(context=BaseError)
def view_exception(exc, request):
r = InquiryResponse()
r['Response Code'] = exc.code
r['Notification Message'] = exc.message
prefix = log_prefix(request)
msg = f'{prefix} Send {r}'
log_web_error(msg)
response = Response(str(r))
response.status_int = 200
return response
class View(BaseView):
def get_db_session(self): # Override
return get_db_session()
def create_iso_log(self, iso, rpc):
conf = self.get_iso_conf()
iso_log = Log(
mti=iso.getMTI(),
rpc_id=rpc.id,
ip=conf['ip'],
conf_name=conf['name'])
for bit in iso.get_bit_definition():
try:
value = iso.getBit(bit)
except BitNotSet:
continue
field = 'bit_{}'.format(str(bit).zfill(3))
setattr(iso_log, field, value)
try:
data = iso.getBit(62)
profile = FixLength(INVOICE_PROFILE)
profile.set_raw(data)
iso_log.bit_062_data = profile.to_dict()
except BitNotSet:
pass
return iso_log
def before_send_iso(self, data, inq, pay, iso_req):
DBSession = get_db_session()
row = self.create_db_log(data, inq, pay)
row.stan = iso_req.get_stan()
with transaction.manager:
DBSession.add(row)
DBSession.flush()
DBSession.expunge_all() # Agar dapat row.id
iso_log = self.create_iso_log(iso_req, row)
DBSession.add(iso_log)
return row
def after_send_iso(self, data, inq, pay, row, iso_resp):
DBSession = get_db_session()
iso_log = self.create_iso_log(iso_resp, row)
iso_data = iso_to_dict(iso_resp)
web_data = get_template_response(data)
if iso_data[39] == '00':
web_data['Response Code'] = '00'
web_data['Notification Message'] = get_ok_notify(data)
if is_inquiry(data):
web_data['Bill Ref'] = str(row.id)
if iso_data.get(62):
profile = FixLength(INVOICE_PROFILE)
profile.set_raw(iso_data[62])
iso_log.bit_062_data = profile.to_dict()
web_data['Biller Name'] = row.biller_name = \
profile2name(profile)
web_data['Bill Amount'] = iso_data[4].lstrip('0')
if iso_data.get(47):
web_data['Transaction ID'] = row.ntp = iso_data[47] # NTP
err = None
elif iso_data[39] in ['33', '55']:
err = InvoiceIdError()
elif iso_data[39] == '54':
if is_reversal(data):
err = PaymentNotFound()
else:
err = AlreadyPaidError()
elif iso_data[39] == '51':
err = AmountError()
elif iso_data[39] == '91':
err = LinkError()
else:
err = BaseError()
if err:
web_data.from_err(err)
row.resp_msg = web_data['Notification Message']
self.log_send(web_data.values)
row.resp_code = web_data['Response Code']
row.resp_msg = web_data['Notification Message']
with transaction.manager:
DBSession.add(row)
DBSession.add(iso_log)
return web_data
def get_invoice_id(self, data): # Override
s = data['acc_no']
if len(s) == 18:
s += data['msg']
return s
def get_invoice_id_from_inquiry(self, inq):
s = inq.acc_no
if len(s) == 18:
s += inq.msg
return s
def validate_data(self, data):
prefix_found = get_ok_notify(data)
if not prefix_found:
raise InvoiceIdError()
def get_response(self, data): # Override
self.validate_data(data)
p, inq, pay = self.prepare_response(data)
conn = self.get_connection()
method = get_method(data)
iso_func = getattr(conn.job, method)
iso_req = iso_func(p)
row = self.before_send_iso(data, inq, pay, iso_req)
iso_resp = self.web_job(conn, iso_req)
return self.after_send_iso(data, inq, pay, row, iso_resp)
@csv_method(route_name=ROUTE)
def view(self):
return super().view()
# Dipanggil read_conf.py
def init(cfg):
conf.update(cfg)
def str2dict(s):
r = dict()
for token in s.split('\n'):
s = token.strip()
if not s:
continue
key, value = s.split(':')
r[key] = value
return r
# Dipanggil forwarder.py
def pyramid_init(config):
base_pyramid_init(config, conf, ROUTE)
conf['notification_message'] = str2dict(conf['notification_message'])
iso8583_web/scripts/views/linkaja/sambat.py deleted 100644 → 0
import transaction
from pyramid.view import view_config
from deform import ValidationFailure
from opensipkd.string import FixLength
from opensipkd.iso8583.bjb.pbb.structure import INVOICE_PROFILE
from opensipkd.iso8583.bjb.pbb.agratek.models import (
Rpc,
Log,
)
from iso8583_web.scripts.tools import iso_to_dict
from iso8583_web.read_conf import web_path_conf
from iso8583_web.scripts.logger import (
log_web_info,
log_web_error,
)
from . import (
WebJob as BaseWebJob,
View as BaseView,
)
from pyramid_linkaja.exceptions import (
InvoiceIdError,
NeedPostError,
InternalError,
TrxTypeError,
HostError,
AlreadyPaidError,
TimeoutError,
BaseError,
AmountError,
BillRefNotFound,
PaymentNotFound,
LinkError,
)
from pyramid_linkaja.responses import (
InquiryResponse,
PaymentResponse,
get_trx_date,
get_method,
get_template_response,
is_inquiry,
is_payment,
is_reversal,
)
from pyramid_linkaja.decorator import csv_method
from pyramid_linkaja.structure import RENDERER
from pyramid_linkaja.form import get_form
from . import (
get_inquiry,
get_payment,
pyramid_init as base_pyramid_init,
)
ROUTE = 'linkaja-sambat'
conf = dict()
def get_db_session():
return conf['db_session']
class WebJob(BaseWebJob):
def timeout_error(self): # override
return TimeoutError()
def profile2name(profile):
msg = [profile['Nama'].strip()]
fields = [('Tagihan', 'Pok'), ('Denda', 'Den'), ('Discount', 'Disk')]
for field, label in fields:
s = profile[field].strip().lstrip('0')
if s:
msg.append(f'{label} {s}')
fields = ['Nama Kelurahan', 'Nama Kecamatan', 'Lokasi']
for field in fields:
s = profile[field].strip()
if s:
msg.append(s)
return ', '.join(msg)
def get_ok_notify(data):
for key in conf['notification_message']:
if data['acc_no'].find(key) == 0:
return conf['notification_message'][key]
return ''
class View(BaseView):
def get_web_job_cls(self): # Override
return WebJob
def not_found_error(self, hostname): # Override
return HostError(hostname)
def not_running_error(self, hostname): # Override
msg = f'Host {hostname} belum terhubung'
return InternalError(msg, 'Sedang offline')
def create_iso_log(self, iso, rpc):
conf = self.get_iso_conf()
iso_log = Log(
mti=iso.getMTI(),
rpc_id=rpc.id,
ip=conf['ip'],
conf_name=conf['name'])
for bit in iso.get_bit_definition():
try:
value = iso.getBit(bit)
except BitNotSet:
continue
field = 'bit_{}'.format(str(bit).zfill(3))
setattr(iso_log, field, value)
try:
data = iso.getBit(62)
profile = FixLength(INVOICE_PROFILE)
profile.set_raw(data)
iso_log.bit_062_data = profile.to_dict()
except BitNotSet:
pass
return iso_log
def before_send_iso(self, data, inq, pay, iso_req):
DBSession = get_db_session()
web_conf = self.get_web_conf()
row = Rpc(
ip=self.request.client_addr,
conf_name=web_conf['name'],
merchant=data['merchant'],
terminal=data['terminal'],
trx_type=data['trx_type'],
msisdn=data['msisdn'],
acc_no=data['acc_no'],
stan=iso_req.get_stan())
row.trx_date = get_trx_date(data),
if data.get('msg'):
row.msg = data['msg']
if data.get('amount'):
row.amount = int(data['amount'])
if not is_inquiry(data):
row.inquiry_id = inq and inq.id or pay.inquiry_id
row.ntb = data['trx_id']
with transaction.manager:
DBSession.add(row)
DBSession.flush()
DBSession.expunge_all() # Agar dapat row.id
iso_log = self.create_iso_log(iso_req, row)
DBSession.add(iso_log)
return row
def after_send_iso(self, data, inq, pay, row, iso_resp):
DBSession = get_db_session()
iso_log = self.create_iso_log(iso_resp, row)
iso_data = iso_to_dict(iso_resp)
web_data = get_template_response(data)
if iso_data[39] == '00':
web_data['Response Code'] = '00'
web_data['Notification Message'] = get_ok_notify(data)
if is_inquiry(data):
web_data['Bill Ref'] = str(row.id)
if iso_data.get(62):
profile = FixLength(INVOICE_PROFILE)
profile.set_raw(iso_data[62])
iso_log.bit_062_data = profile.to_dict()
web_data['Biller Name'] = row.biller_name = \
profile2name(profile)
web_data['Bill Amount'] = iso_data[4].lstrip('0')
if iso_data.get(47):
web_data['Transaction ID'] = row.ntp = iso_data[47] # NTP
err = None
elif iso_data[39] in ['33', '55']:
err = InvoiceIdError()
elif iso_data[39] == '54':
if is_reversal(data):
err = PaymentNotFound()
else:
err = AlreadyPaidError()
elif iso_data[39] == '51':
err = AmountError()
elif iso_data[39] == '91':
err = LinkError()
else:
err = BaseError()
if err:
web_data.from_err(err)
row.resp_msg = web_data['Notification Message']
self.log_send(web_data.values)
row.resp_code = web_data['Response Code']
row.resp_msg = web_data['Notification Message']
with transaction.manager:
DBSession.add(row)
DBSession.add(iso_log)
return web_data
def get_response(self, data):
p, inq, pay = self.prepare_response(data)
return self.after_send_iso(data, inq, pay, row, iso_resp) #FIXME
@csv_method(route_name=ROUTE)
def view(self):
return super().view()
# Dipanggil read_conf.py
def init(cfg):
conf.update(cfg)
# Dipanggil forwarder.py
def pyramid_init(config):
base_pyramid_init(config, conf, ROUTE)
iso8583_web/scripts/web_client.py deleted 100644 → 0
import sys
import os
import requests
import json
from datetime import datetime
from time import (
sleep,
time,
)
from threading import Thread
from argparse import ArgumentParser
headers = {'content-type': 'application/json'}
threads = dict()
end_threads = list()
durations = dict()
json_responses = dict()
server_info = dict()
default_url = 'http://localhost:7000/rpc'
default_count = 1
help_url = 'default ' + default_url
help_count = 'default {}'.format(default_count)
help_invoice_id = 'wajib saat --payment dan --reversal'
help_amount = 'wajib saat --payment dan --reversal'
help_ntb = 'opsional saat --payment, wajib saat --reversal'
help_stan = 'opsional saat --payment, wajib saat --reversal'
help_bit = 'bit tambahan, contoh: --bit=42:TOKOPEDIA'
help_conf = 'konfigurasi tambahan, contoh untuk multi: --conf=pajak:bphtb'
def error(s):
print('ERROR: {}'.format(s))
sys.exit()
def log_info(s):
t = datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')
t = t[:-3]
msg = '{} {}'.format(t, s)
print(msg)
def get_option(argv):
parser = ArgumentParser()
parser.add_argument('--url', default=default_url, help=help_url)
parser.add_argument(
'--count', type=int, default=default_count, help=help_count)
parser.add_argument('--invoice-id', help=help_invoice_id)
parser.add_argument('--payment', action='store_true')
parser.add_argument('--reversal', action='store_true')
parser.add_argument('--amount', type=int, help=help_amount)
parser.add_argument('--ntb', help=help_ntb)
parser.add_argument('--stan', help=help_stan)
parser.add_argument('--bit', help=help_bit)
parser.add_argument('--conf', help=help_conf)
return parser.parse_args(argv)
def send(p):
url = server_info['url']
key = p['id']
log_info('Request: {}'.format(p))
start = time()
try:
resp = requests.post(url, data=json.dumps(p), headers=headers)
durations[key] = time() - start
if resp.status_code == 200:
json_resp = resp.json()
log_info('Response: {}'.format(json_resp))
json_responses[key] = json_resp
else:
log_info('Status Code: {}'.format(resp.status_code))
log_info('Body: {}'.format([resp.text]))
json_responses[key] = dict(fatal=resp.text)
except requests.exceptions.ConnectionError as e:
durations[key] = time() - start
log_info('Response: {}'.format(e))
json_responses[key] = dict(fatal=e)
except json.decoder.JSONDecodeError as e:
durations[key] = time() - start
log_info('Body: {}'.format([resp.text]))
log_info('Response: {}'.format(e))
json_responses[key] = dict(fatal=e)
finally:
end_threads.append(key)
def stan_from_result(result):
if result['code'] == 0:
return result['data']['11']
return '-'
def show_errors(errors):
if errors:
for err, count in errors.items():
log_info('{} {}'.format(err, count))
else:
log_info('Tidak ada yang gagal')
def show_durations():
key_fastest = None
key_slowest = None
total_duration = 0
messages = dict()
errors = dict()
for key in durations:
duration = durations[key]
msg = 'thread {} {} detik'.format(key, duration)
resp = json_responses[key]
if 'fatal' in resp:
errors['fatal'] = resp['fatal']
elif 'error' in resp:
result = resp['error']
msg = '{} {}'.format(msg, result['message'])
err = result['message']
if err in errors:
errors[err] += 1
else:
errors[err] = 1
else:
result = resp['result']
stan = stan_from_result(result)
msg = '{} stan {}'.format(msg, stan)
messages[key] = msg
if key_fastest:
if duration < durations[key_fastest]:
key_fastest = key
else:
key_fastest = key
if key_slowest:
if duration > durations[key_slowest]:
key_slowest = key
else:
key_slowest = key
total_duration += duration
log_info(msg)
if key_fastest != key_slowest:
log_info('Tercepat {}'.format(messages[key_fastest]))
log_info('Terlama {}'.format(messages[key_slowest]))
log_info('Rerata {} detik / request'.format(
total_duration/len(durations)))
show_errors(errors)
class App:
def __init__(self, argv):
self.option = get_option(argv)
server_info['url'] = self.option.url
def create_thread(self, data):
thread = Thread(target=send, args=[data])
# Exit the server thread when the main thread terminates
thread.daemon = True
thread_id = data['id']
threads[thread_id] = thread
thread.start()
def get_invoice_ids(self):
if not os.path.exists(self.option.invoice_id):
return [self.option.invoice_id]
r = []
with open(self.option.invoice_id) as f:
for line in f.readlines():
invoice_id = line.rstrip()
r += [invoice_id]
return r
def get_method(self):
if self.option.payment:
return 'payment'
if self.option.reversal:
return 'reversal'
return 'inquiry'
def get_transaction(self, invoice_id):
def required(name, default=None):
value = getattr(self.option, name)
if not value and not default:
error('--{} harus diisi'.format(name))
p[name] = value or default
p = dict(invoice_id=invoice_id)
if self.option.payment or self.option.reversal:
required('amount')
if self.option.payment:
required('ntb', datetime.now().strftime('%m%d%H%M%S'))
required('stan', datetime.now().strftime('%H%M%S'))
else:
required('ntb')
required('stan')
if self.option.bit:
bits = dict()
for t in self.option.bit.split(','):
bit, value = t.split(':')
bits[bit] = value
p['bits'] = bits
if self.option.conf:
conf = dict()
for t in self.option.conf.split(','):
key, val = t.split(':')
conf[key] = val
p['conf'] = conf
return p
def run_transaction(self):
method = self.get_method()
thread_id = 0
for i in range(self.option.count):
for invoice_id in self.get_invoice_ids():
thread_id += 1
p = self.get_transaction(invoice_id)
data = dict(
id=thread_id, method=method, params=[p],
jsonrpc='2.0')
self.create_thread(data)
def run_echo(self):
for thread_id in range(1, self.option.count+1):
p = dict(id=thread_id)
data = dict(id=thread_id, method='echo', params=[p], jsonrpc='2.0')
self.create_thread(dict(data))
def run(self):
p = dict()
if self.option.invoice_id:
self.run_transaction()
else:
self.run_echo()
while threads:
if not end_threads:
continue
i = end_threads[0]
if i in threads:
thread = threads[i]
thread.join()
del threads[i]
index = end_threads.index(i)
del end_threads[index]
show_durations()
def main(argv=sys.argv[1:]):
app = App(argv)
app.run()
iso8583_web/scripts/web_client_linkaja.py deleted 100644 → 0
import sys
import os
import requests
from datetime import datetime
from time import (
sleep,
time,
)
from threading import Thread
from argparse import ArgumentParser
from pyramid_linkaja.responses import (
InquiryResponse,
PaymentResponse,
)
headers = {'content-type': 'application/x-www-form-urlencoded'}
threads = dict()
end_threads = list()
durations = dict()
csv_responses = dict()
server_info = dict()
default_url = 'http://localhost:7000/linkaja'
default_count = 1
default_merchant = 'ldmjakarta1'
default_terminal = 'Terminal Name'
default_pwd = 'ldmjkt1pass'
default_msisdn = '628111234567'
default_timeout = 35
help_url = 'default ' + default_url
help_count = f'default {default_count}'
help_amount = 'wajib saat --payment dan --reversal'
help_bill_ref = 'wajib saat payment dan reversal, '\
'diperoleh dari inquiry response'
help_trx_id = 'Nomor Transaksi Bank, '\
'opsional saat --payment, wajib saat --reversal'
help_merchant = 'default ' + default_merchant
help_timeout = f'default {default_timeout} detik'
ERRORS = [
'Connection refused',
]
def error(s):
print('ERROR: {}'.format(s))
sys.exit()
def log_info(s):
t = datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')
t = t[:-3]
msg = '{} {}'.format(t, s)
print(msg)
def get_option(argv):
parser = ArgumentParser()
parser.add_argument('--url', default=default_url, help=help_url)
parser.add_argument(
'--count', type=int, default=default_count, help=help_count)
parser.add_argument('--invoice-id', required=True)
parser.add_argument('--payment', action='store_true')
parser.add_argument('--reversal', action='store_true')
parser.add_argument('--amount', type=int, help=help_amount)
parser.add_argument('--bill-ref', help=help_bill_ref)
parser.add_argument('--trx-id', help=help_trx_id)
parser.add_argument(
'--merchant', default=default_merchant, help=help_merchant)
parser.add_argument('--terminal', default=default_terminal)
parser.add_argument('--pwd', default=default_pwd)
parser.add_argument('--msisdn', default=default_msisdn)
parser.add_argument('--msg', default='')
parser.add_argument(
'--timeout', help=help_timeout, default=default_timeout, type=int)
return parser.parse_args(argv)
def send(thread_id, p):
url = server_info['url']
timeout = server_info['timeout']
log_info('Request: {}'.format(p))
start = time()
try:
resp = requests.post(url, data=p, headers=headers, timeout=timeout)
durations[thread_id] = time() - start
data = p['trx_type'] == '021' and InquiryResponse() or \
PaymentResponse()
if resp.status_code == 200:
data.from_raw(resp.text)
log_info('Response {}: {} -> {}'.format(
resp.status_code, [resp.text], data.values))
csv_responses[thread_id] = resp
except requests.exceptions.ConnectionError as e:
durations[thread_id] = time() - start
log_info('Response: {}'.format(e))
csv_responses[thread_id] = dict(fatal=e)
except requests.exceptions.ReadTimeout as e:
durations[thread_id] = time() - start
log_info('Response: {}'.format(e))
csv_responses[thread_id] = dict(fatal=e)
finally:
end_threads.append(thread_id)
def show_errors(errors):
if errors:
for err, count in errors.items():
log_info('{} {}'.format(err, count))
else:
log_info('Tidak ada yang gagal')
def nice_error(s):
for msg in ERRORS:
if s.find(msg) > -1:
return msg
return s
def show_durations():
tid_fastest = tid_slowest = None
total_duration = 0
messages = dict()
errors = dict()
for tid in durations:
duration = durations[tid]
resp = csv_responses.get(tid)
if resp:
err = None
if 'fatal' in resp:
err = msg = nice_error(str(resp['fatal']))
elif resp.status_code == 200:
messages[tid] = msg = resp.text.strip()
if tid_fastest:
if duration < durations[tid_fastest]:
tid_fastest = tid
else:
tid_fastest = tid
if tid_slowest:
if duration > durations[tid_slowest]:
tid_slowest = tid
else:
tid_slowest = tid
total_duration += duration
else:
err = msg = resp.text.split('\n')[0].strip()
else:
err = msg = 'KOSONG'
if err:
if err in errors:
errors[err] += 1
else:
errors[err] = 1
log_info('thread {} {} detik {}'.format(tid, duration, msg))
if tid_fastest != tid_slowest:
log_info('Tercepat {}'.format(messages[tid_fastest]))
log_info('Terlama {}'.format(messages[tid_slowest]))
log_info('Rerata {} detik / request'.format(
total_duration/len(durations)))
show_errors(errors)
class App:
def __init__(self, argv):
self.option = get_option(argv)
server_info['url'] = self.option.url
server_info['timeout'] = self.option.timeout
def create_thread(self, thread_id, data):
thread = Thread(target=send, args=[thread_id, data])
# Exit the server thread when the main thread terminates
thread.daemon = True
threads[thread_id] = thread
thread.start()
def get_invoice_ids(self):
if not os.path.exists(self.option.invoice_id):
return [self.option.invoice_id]
r = []
with open(self.option.invoice_id) as f:
for line in f.readlines():
invoice_id = line.rstrip()
r += [invoice_id]
return r
def get_method(self):
if self.option.payment:
return '022'
if self.option.reversal:
return '023'
return '021'
def get_transaction(self, invoice_id):
def required(name, default=None):
value = getattr(self.option, name)
if not value and not default:
error('--{} harus diisi'.format(name.replace('_', '-')))
p[name] = value or default
p = dict(
merchant=self.option.merchant,
terminal=self.option.terminal,
pwd=self.option.pwd,
msisdn=self.option.msisdn,
acc_no=self.option.invoice_id,
trx_date=datetime.now().strftime('%Y%m%d%H%M%S'),
msg=self.option.msg)
p['trx_type'] = self.get_method()
if self.option.payment or self.option.reversal:
required('amount')
required('bill_ref')
if self.option.payment:
required('trx_id', datetime.now().strftime('%m%d%H%M%S'))
else:
required('trx_id')
return p
def run_transaction(self):
thread_id = 0
for i in range(self.option.count):
for invoice_id in self.get_invoice_ids():
thread_id += 1
data = self.get_transaction(invoice_id)
self.create_thread(thread_id, data)
def run(self):
self.run_transaction()
while threads:
if not end_threads:
continue
i = end_threads[0]
if i in threads:
thread = threads[i]
thread.join()
del threads[i]
index = end_threads.index(i)
del end_threads[index]
show_durations()
def main(argv=sys.argv[1:]):
app = App(argv)
app.run()
iso8583_web/security.py
from .models import DBSession
from .models.ziggurat import (
User,
UserGroup,
......@@ -6,14 +5,14 @@ from .models.ziggurat import (
def group_finder(login, request):
q = DBSession.query(User).filter_by(id=login)
q = request.dbsession.query(User).filter_by(id=login)
u = q.first()
if not u or not u.status:
return # None means logout
r = []
q = DBSession.query(UserGroup).filter_by(user_id=u.id)
q = request.dbsession.query(UserGroup).filter_by(user_id=u.id)
for ug in q:
acl_name = 'group:{gid}'.format(gid=ug.group_id)
acl_name = f'group:{ug.group_id}'
r.append(acl_name)
return r
......@@ -21,5 +20,5 @@ def group_finder(login, request):
def get_user(request):
uid = request.authenticated_userid
if uid:
q = DBSession.query(User).filter_by(id=uid)
q = request.dbsession.query(User).filter_by(id=uid)
return q.first()
iso8583_web/tools/conf.py 0 → 100644
from ..models.conf import Conf
class QConf:
def __init__(self, db_session):
self.db_session = db_session
def get(self, key):
q = self.db_session.query(Conf).filter_by(nama=key)
return q.first()
def get_value(self, key):
row = self.get(key)
return row.nilai
iso8583_web/tools/string.py
......@@ -16,8 +16,7 @@ def to_str(v):
return dmy(v)
if v == 0:
return '0'
if isinstance(v, str) or \
(sys.version_info.major == 2 and isinstance(v, unicode)):
if isinstance(v, str):
return v.strip()
elif isinstance(v, bool):
return v and '1' or '0'
......@@ -30,5 +29,3 @@ def dict_to_str(d):
val = d[key]
r[key] = to_str(val)
return r
iso8583_web/tools/waktu.py
import calendar
import calendar
from datetime import (
date,
datetime,
......@@ -12,13 +12,14 @@ def get_timezone():
return pytz.timezone(settings['timezone'])
def create_datetime(year, month, day, hour=0, minute=7, second=0,
microsecond=0):
tz = get_timezone()
return datetime(year, month, day, hour, minute, second,
microsecond, tzinfo=tz)
def create_datetime(
year, month, day, hour=0, minute=7, second=0, microsecond=0):
tz = get_timezone()
return datetime(
year, month, day, hour, minute, second, microsecond, tzinfo=tz)
def create_date(year, month, day):
def create_date(year, month, day):
return create_datetime(year, month, day)
......@@ -28,30 +29,30 @@ def as_timezone(tz_date):
tz_date = create_datetime(tz_date.year, tz_date.month, tz_date.day,
tz_date.hour, tz_date.minute, tz_date.second,
tz_date.microsecond)
return tz_date.astimezone(localtz)
return tz_date.astimezone(localtz)
def create_now():
tz = get_timezone()
return datetime.now(tz)
def date_from_str(value):
separator = None
value = value.split()[0] # dd-mm-yyyy HH:MM:SS
value = value.split()[0] # dd-mm-yyyy HH:MM:SS
for s in ['-', '/']:
if value.find(s) > -1:
separator = s
break
break
if separator:
t = map(lambda x: int(x), value.split(separator))
y, m, d = t[2], t[1], t[0]
if d > 999: # yyyy-mm-dd
if d > 999: # yyyy-mm-dd
y, d = d, y
else:
y, m, d = int(value[:4]), int(value[4:6]), int(value[6:])
return date(y, m, d)
return date(y, m, d)
def split_time(s):
t = s.split(':') # HH:MM:SS
......@@ -75,7 +76,7 @@ def split_date(s):
for sep in ['-', '/']:
if s.find(sep) > -1:
separator = sep
break
break
if separator:
d, m, y = [int(x) for x in s.split(separator)]
if d > 999: # yyyy-mm-dd
......@@ -86,14 +87,14 @@ def split_date(s):
def split_datetime(s):
t = s.split() # dd-mm-yyyy HH:MM:SS
year, month, day = split_date(t[0])
t = s.split() # dd-mm-yyyy HH:MM:SS
year, month, day = split_date(t[0])
if t[1:]:
hours, minutes, seconds = split_time(t[1])
else:
hours = minutes = seconds = 0
return year, month, day, hours, minutes, seconds
def date_from_str(s):
y, m, d, hh, mm, ss = split_datetime(s)
......
iso8583_web/views/group.py
......@@ -12,7 +12,6 @@ from deform.widget import (
HiddenWidget,
CheckboxChoiceWidget,
)
from ..models import DBSession
from ..models.ziggurat import (
Group,
GroupPermission,
......@@ -22,20 +21,22 @@ from ..models.ziggurat import (
_ = TranslationStringFactory('user')
########
########
# List #
########
########
@view_config(
route_name='group', renderer='templates/group/list.pt',
permission='user-edit')
def view_list(request):
q = DBSession.query(Group).order_by(Group.group_name)
q = request.dbsession.query(Group).order_by(Group.group_name)
return dict(groups=q, title=_('Groups'))
#######
#######
# Add #
#######
def clean_name(s):
s = s.strip()
while s.find(' ') > -1:
......@@ -44,15 +45,16 @@ def clean_name(s):
class GroupNameValidator:
def __init__(self, group):
self.group = group
def __init__(self, kw):
self.group = kw['group']
self.db_session = kw['request'].dbsession
def __call__(self, node, value):
group_name = clean_name(value)
if self.group and self.group.group_name.lower() == group_name.lower():
return
q = DBSession.query(Group).\
filter(Group.group_name.ilike(group_name))
q = self.db_session.query(Group).filter(
Group.group_name.ilike(group_name))
found = q.first()
if not found:
return
......@@ -66,7 +68,7 @@ class GroupNameValidator:
@colander.deferred
def deferred_group_name_validator(node, kw):
return GroupNameValidator(kw['group'])
return GroupNameValidator(kw)
@colander.deferred
......@@ -92,12 +94,13 @@ class AddSchema(colander.Schema):
class EditSchema(AddSchema):
id = colander.SchemaNode(
colander.String(), widget=HiddenWidget(readonly=True))
PERMISSIONS = [
('edit-user', _('User management'))
]
def get_form(request, class_form, group=None):
schema = class_form()
schema = schema.bind(permission_list=PERMISSIONS, group=group)
......@@ -107,18 +110,18 @@ def get_form(request, class_form, group=None):
return Form(schema, buttons=buttons)
def insert(values):
def insert(db_session, values):
group = Group()
group.group_name = clean_name(values['group_name'])
group.description = values['description']
DBSession.add(group)
DBSession.flush()
db_session.add(group)
db_session.flush()
for perm_name in values['permissions']:
gp = GroupPermission()
gp.group_id = group.id
gp.perm_name = perm_name
DBSession.add(gp)
return group
db_session.add(gp)
return group
@view_config(
......@@ -138,7 +141,7 @@ def view_add(request):
except ValidationFailure:
resp['form'] = form.render()
return resp
group = insert(dict(c.items()))
group = insert(request.dbsession, dict(c.items()))
data = dict(group_name=group.group_name)
ts = _(
'group-added',
......@@ -147,25 +150,27 @@ def view_add(request):
request.session.flash(ts)
return HTTPFound(location=request.route_url('group'))
########
# Edit #
########
def group_permission_set(group):
q = DBSession.query(GroupPermission).filter_by(group_id=group.id)
def group_permission_set(db_session, group):
q = db_session.query(GroupPermission).filter_by(group_id=group.id)
r = []
for gp in q:
r.append(gp.perm_name)
return set(r)
def update(group, values):
def update(db_session, group, values):
group.group_name = clean_name(values['group_name'])
group.description = values['description']
DBSession.add(group)
existing = group_permission_set(group)
db_session.add(group)
existing = group_permission_set(db_session, group)
unused = existing - values['permissions']
if unused:
q = DBSession.query(GroupPermission).filter_by(group_id=group.id).\
q = db_session.query(GroupPermission).filter_by(group_id=group.id).\
filter(GroupPermission.perm_name.in_(unused))
q.delete(synchronize_session=False)
new = values['permissions'] - existing
......@@ -173,14 +178,14 @@ def update(group, values):
gp = GroupPermission()
gp.group_id = group.id
gp.perm_name = perm_name
DBSession.add(gp)
db_session.add(gp)
@view_config(
route_name='group-edit', renderer='templates/group/edit.pt',
permission='user-edit')
def view_edit(request):
q = DBSession.query(Group).filter_by(id=request.matchdict['id'])
q = request.dbsession.query(Group).filter_by(id=request.matchdict['id'])
group = q.first()
if not group:
return HTTPNotFound()
......@@ -200,21 +205,24 @@ def view_edit(request):
except ValidationFailure:
resp['form'] = form.render()
return resp
update(group, dict(c.items()))
update(request.dbsession, group, dict(c.items()))
data = dict(group_name=group.group_name)
ts = _('group-updated', default='${group_name} group profile updated', mapping=data)
ts = _(
'group-updated', default='${group_name} group profile updated',
mapping=data)
request.session.flash(ts)
return HTTPFound(location=request.route_url('group'))
##########
# Delete #
##########
##########
@view_config(
route_name='group-delete', renderer='templates/group/delete.pt',
permission='user-edit')
def view_delete(request):
q = DBSession.query(Group).filter_by(id=request.matchdict['id'])
q = request.dbsession.query(Group).filter_by(id=request.matchdict['id'])
group = q.first()
if not group:
return HTTPNotFound()
......
iso8583_web/views/log.py deleted 100644 → 0
import os
from datetime import timedelta
from pkg_resources import resource_filename
from sqlalchemy import func
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPFound
from paginate_sqlalchemy import SqlalchemyOrmPage
import colander
from deform import (
Form,
widget,
Button,
ZPTRendererFactory,
ValidationFailure,
)
from ..models import DBSession
from ..models.log import (
Log,
Jenis,
)
from ..tools.waktu import (
datetime_from_str,
create_now,
)
from ..tools.string import dict_to_str
LIMIT = 100
def get_page(request):
page = request.GET.get('page')
return page and int(page) or 1
def get_jenis_list():
q = DBSession.query(Jenis).order_by(Jenis.nama)
values = []
for row in q:
row = (str(row.id), row.nama)
values.append(row)
return values
@colander.deferred
def jenis_widget(node, kw):
return widget.SelectWidget(values=kw['jenis_list'])
class FilterSchema(colander.Schema):
awal = colander.SchemaNode(colander.String())
akhir = colander.SchemaNode(colander.String(), missing=colander.drop)
jenis = colander.SchemaNode(
colander.String(), missing=colander.drop, widget=jenis_widget)
keterangan = colander.SchemaNode(colander.String(), missing=colander.drop)
deform_templates = resource_filename('deform', 'templates')
here = os.path.abspath(os.path.dirname(__file__))
my_templates = os.path.join(here, 'templates', 'log')
search_path = [my_templates, deform_templates]
my_renderer = ZPTRendererFactory(search_path)
def get_form():
schema = FilterSchema()
schema = schema.bind(jenis_list=get_jenis_list())
btn_lihat = Button('lihat', 'Lihat')
btn_terbaru = Button('terbaru', 'Terbaru')
buttons = (btn_lihat, btn_terbaru)
return Form(schema, buttons=buttons, renderer=my_renderer)
@view_config(
route_name='log', renderer='templates/log/list.pt', permission='view')
def view_list(request):
def url_maker(page):
d['page'] = page
d['lihat'] = 1
return request.route_url('log', _query=d)
form = get_form()
resp = dict(title='Log file')
if request.POST:
items = request.POST.items()
try:
c = form.validate(items)
except ValidationFailure as e:
resp['form'] = e.render()
return resp
p = dict(c.items())
if 'terbaru' in request.POST:
p['awal'] = default_awal(p['jenis'])
p['lihat'] = 1
return route_list(request, p)
if 'awal' not in request.GET:
p = default_filter()
return route_list(request, p)
p = get_filter(request)
d = dict_to_str(p)
resp['form'] = form.render(appstruct=d)
if 'lihat' not in request.GET:
return resp
q = get_query(p)
resp['count'] = count = q.count()
q = q.order_by(Log.tgl)
page = get_page(request)
resp['rows'] = SqlalchemyOrmPage(
q, page=page, items_per_page=10, item_count=count,
url_maker=url_maker)
return resp
def default_filter():
jenis = get_last_jenis_trx()
awal = default_awal(jenis)
p = dict(jenis=jenis, awal=awal)
return p
def default_awal(jenis):
q = DBSession.query(Log).filter_by(jenis_id=jenis)
jml = q.count()
if jml >= LIMIT:
offset = jml - LIMIT
else:
offset = 0
q = DBSession.query(Log.tgl).filter_by(jenis_id=jenis).\
order_by(Log.tgl).offset(offset).limit(1)
row = q.first()
if row:
return row.tgl
return create_now()
def get_filter(request):
p = dict(jenis=int(request.params.get('jenis')))
p['awal'] = datetime_from_str(request.params['awal'])
akhir = request.params.get('akhir')
if akhir:
p['akhir'] = datetime_from_str(akhir)
keterangan = request.params.get('keterangan')
if keterangan:
p['keterangan'] = keterangan
return p
def get_query(p):
q = DBSession.query(Log).filter_by(jenis_id=p['jenis']).\
filter(Log.tgl >= p['awal'])
if 'akhir' in p:
q = q.filter(Log.tgl <= p['akhir'])
if 'keterangan' in p:
q = q.filter(Log.line.ilike('%'+p['keterangan']+'%'))
return q
def get_last_jenis_trx():
q = DBSession.query(Log).order_by(Log.tgl.desc())
r = q.first()
if r:
return r.jenis_id
q = DBSession.query(Jenis).order_by(Jenis.id)
r = q.first()
return r.id
def route_list(request, p=dict()):
q = dict_to_str(p)
return HTTPFound(location=request.route_url('log', _query=q))
iso8583_web/views/login.py
......@@ -27,7 +27,6 @@ from ziggurat_foundations.models.services.user import UserService
from pyramid_mailer.message import Message
from ..tools.waktu import create_now
from ..tools.this_framework import get_settings
from ..models import DBSession
from ..models.ziggurat import User
......@@ -46,41 +45,42 @@ class Login(colander.Schema):
def login_validator(form, value):
user = form.user
if not user or \
not user.status or \
not user.user_password or \
not UserService.check_password(user, value['password']):
not user.status or \
not user.user_password or \
not UserService.check_password(user, value['password']):
raise colander.Invalid(form, _('Login failed'))
def login_ok(request, user, route='log'):
headers = remember(request, user.id)
user.last_login_date = create_now()
DBSession.add(user)
request.dbsession.add(user)
return HTTPFound(location=request.route_url(route), headers=headers)
def get_user_by_identity(request):
identity = request.POST.get('username')
if identity.find('@') > -1:
q = DBSession.query(User).filter_by(email=identity)
q = request.dbsession.query(User).filter_by(email=identity)
else:
q = DBSession.query(User).filter_by(user_name=identity)
q = request.dbsession.query(User).filter_by(user_name=identity)
return q.first()
one_hour = timedelta(1/24)
two_minutes = timedelta(1/24/60)
def login_by_code(request):
q = DBSession.query(User).filter_by(security_code=request.GET['code'])
q = request.dbsession.query(User).filter_by(
security_code=request.GET['code'])
user = q.first()
if not user or \
create_now() - user.security_code_date > one_hour:
create_now() - user.security_code_date > one_hour:
return HTTPFound(location=request.route_url('login-by-code-failed'))
user.security_code = None
DBSession.add(user)
DBSession.flush()
request.dbsession.add(user)
request.dbsession.flush()
return login_ok(request, user, 'change-password')
......@@ -103,7 +103,7 @@ def view_login(request):
schema = Login(validator=login_validator)
btn_submit = Button('submit', _('Submit'))
form = Form(schema, buttons=(btn_submit,))
if 'submit' not in request.POST:
if 'submit' not in request.POST:
resp['form'] = form.render()
return resp
controls = request.POST.items()
......@@ -119,11 +119,12 @@ def view_login(request):
@view_config(route_name='logout')
def view_logout(request):
headers = forget(request)
return HTTPFound(location = request.route_url('home'),
headers = headers)
return HTTPFound(location=request.route_url('home'), headers=headers)
@view_config(route_name='login-by-code-failed', renderer='templates/login-by-code-failed.pt')
@view_config(
route_name='login-by-code-failed',
renderer='templates/login-by-code-failed.pt')
def view_login_by_code_failed(request):
return dict(title='Login by code failed')
......@@ -143,7 +144,7 @@ class ChangePassword(colander.Schema):
def password_validator(form, value):
if value['new_password'] != value['retype_password']:
raise colander.Invalid(form, _('Retype mismatch'))
@view_config(
route_name='change-password', renderer='templates/change-password.pt',
......@@ -167,15 +168,16 @@ def view_change_password(request):
resp['form'] = form.render()
return resp
UserService.set_password(request.user, c['new_password'])
DBSession.add(request.user)
request.dbsession.add(request.user)
return HTTPFound(location=request.route_url('change-password-done'))
@view_config(
route_name='change-password-done',
renderer='templates/change-password-done.pt', permission='view')
def view_change_password_done(request):
return dict(title=_('Change password'))
##################
# Reset password #
......@@ -185,7 +187,7 @@ class ResetPassword(colander.Schema):
colander.String(), title=_('Email'),
description=_(
'email-reset-password',
default='Enter your email address and we will send you '\
default='Enter your email address and we will send you '
'a link to reset your password.')
)
......@@ -201,7 +203,7 @@ def security_code_age(user):
def send_email_security_code(
request, user, time_remain, subject, body_msg_id, body_default_file):
request, user, time_remain, subject, body_msg_id, body_default_file):
settings = get_settings()
up = urlparse(request.url)
url = '{}://{}/login?code={}'.format(
......@@ -223,18 +225,19 @@ def send_email_security_code(
mailer.send(message)
def regenerate_security_code(user):
def regenerate_security_code(db_session, user):
age = security_code_age(user)
remain = one_hour - age
if user.security_code and age < one_hour and remain > two_minutes:
return remain
UserService.regenerate_security_code(user)
user.security_code_date = create_now()
DBSession.add(user)
db_session.add(user)
return one_hour
@view_config(route_name='reset-password', renderer='templates/reset-password.pt')
@view_config(
route_name='reset-password', renderer='templates/reset-password.pt')
def view_reset_password(request):
if authenticated_userid(request):
return HTTPFound(location=request.route_url('home'))
......@@ -242,17 +245,17 @@ def view_reset_password(request):
schema = ResetPassword(validator=reset_password_validator)
btn_submit = Button('submit', _('Send password reset email'))
form = Form(schema, buttons=(btn_submit,))
if 'submit' in request.POST:
if 'submit' in request.POST:
controls = request.POST.items()
identity = request.POST.get('email')
q = DBSession.query(User).filter_by(email=identity)
q = request.dbsession.query(User).filter_by(email=identity)
schema.user = user = q.first()
try:
c = form.validate(controls)
except ValidationFailure:
resp['form'] = form.render()
return resp
remain = regenerate_security_code(user)
remain = regenerate_security_code(request.dbsession, user)
send_email_security_code(
request, user, remain, 'Reset password', 'reset-password-body',
'reset-password-body.tpl')
......@@ -262,7 +265,7 @@ def view_reset_password(request):
@view_config(
route_name='reset-password-sent', renderer='templates/reset-password-sent.pt')
route_name='reset-password-sent',
renderer='templates/reset-password-sent.pt')
def view_reset_password_sent(request):
return dict(title=_('Reset password'))
iso8583_web/views/user.py
......@@ -19,7 +19,6 @@ from deform.widget import (
PasswordWidget,
HiddenWidget,
)
from ..models import DBSession
from ..models.ziggurat import (
User,
Group,
......@@ -36,14 +35,14 @@ from .login import (
_ = TranslationStringFactory('user')
########
########
# List #
########
########
def query_filter(request, q):
return q.filter(
User.id==UserGroup.user_id,
UserGroup.group_id==request.GET['gid'])
User.id == UserGroup.user_id,
UserGroup.group_id == request.GET['gid'])
@view_config(
......@@ -58,27 +57,28 @@ def view_list(request):
int(request.GET['gid'])
except ValueError:
return HTTPNotFound()
q_count = DBSession.query(func.count())
q_count = request.dbsession.query(func.count())
q_count = query_filter(request, q_count)
count = q_count.scalar()
if count:
q_user = DBSession.query(User)
q_user = request.dbsession.query(User)
q_user = query_filter(request, q_user)
else:
q_count = DBSession.query(func.count(User.id))
q_count = request.dbsession.query(func.count(User.id))
count = q_count.scalar()
if count:
q_user = DBSession.query(User)
q_group = DBSession.query(Group).order_by(Group.group_name)
q_user = request.dbsession.query(User)
q_group = request.dbsession.query(Group).order_by(Group.group_name)
resp = dict(title=_('Users'), count=count, groups=q_group)
if count:
resp['users'] = q_user.order_by(User.email)
return resp
#######
#######
# Add #
#######
@colander.deferred
def deferred_status(node, kw):
values = kw.get('status_list', [])
......@@ -97,7 +97,9 @@ class Validator:
class EmailValidator(colander.Email, Validator):
def __init__(self, user):
def __init__(self, kw):
user = kw['user']
self.db_session = kw['request'].dbsession
colander.Email.__init__(self)
Validator.__init__(self, user)
......@@ -107,7 +109,7 @@ class EmailValidator(colander.Email, Validator):
email = value.lower()
if self.user and self.user.email == email:
return
q = DBSession.query(User).filter_by(email=email)
q = self.db_session.query(User).filter_by(email=email)
found = q.first()
if not found:
return
......@@ -122,7 +124,11 @@ class EmailValidator(colander.Email, Validator):
REGEX_ONLY_CONTAIN = re.compile('([a-z0-9-]*)')
REGEX_BEGIN_END_ALPHANUMERIC = re.compile('^[a-z0-9]+(?:[-][a-z0-9]+)*$')
class UsernameValidator(Validator):
def __init__(self, kw):
self.db_session = kw['request'].dbsession
def __call__(self, node, value):
username = value.lower()
if self.user and self.user.user_name == username:
......@@ -139,7 +145,7 @@ class UsernameValidator(Validator):
'username-first-end-alphanumeric',
default='Only a-z or 0-9 at the start and end')
raise colander.Invalid(node, ts)
q = DBSession.query(User).filter_by(user_name=username)
q = self.db_session.query(User).filter_by(user_name=username)
found = q.first()
if not found:
return
......@@ -153,19 +159,20 @@ class UsernameValidator(Validator):
@colander.deferred
def deferred_email_validator(node, kw):
return EmailValidator(kw['user'])
return EmailValidator(kw)
@colander.deferred
def deferred_username_validator(node, kw):
return UsernameValidator(kw['user'])
return UsernameValidator(kw)
class AddSchema(colander.Schema):
email = colander.SchemaNode(
colander.String(), title=_('Email'),
validator=deferred_email_validator)
user_name = colander.SchemaNode(colander.String(), title=_('Username'),
user_name = colander.SchemaNode(
colander.String(), title=_('Username'),
validator=deferred_username_validator)
groups = colander.SchemaNode(
colander.Set(), widget=deferred_group, title=_('Group'))
......@@ -177,20 +184,20 @@ class EditSchema(AddSchema):
widget=HiddenWidget(readonly=True))
status = colander.SchemaNode(
colander.String(), widget=deferred_status, title=_('Status'))
class MyEditSchema(AddSchema):
id = colander.SchemaNode(
colander.String(), missing=colander.drop,
widget=HiddenWidget(readonly=True))
def get_form(request, class_form, user=None):
status_list = (
(1, _('Active')),
(0, _('Inactive')))
group_list = []
q = DBSession.query(Group).order_by(Group.group_name)
q = request.dbsession.query(Group).order_by(Group.group_name)
for row in q:
group = (str(row.id), _(row.group_name))
group_list.append(group)
......@@ -202,18 +209,18 @@ def get_form(request, class_form, user=None):
return Form(schema, buttons=(btn_save, btn_cancel))
def add_member_count(gid):
q = DBSession.query(Group).filter_by(id=gid)
def add_member_count(db_session, gid):
q = db_session.query(Group).filter_by(id=gid)
group = q.first()
group.member_count += 1
DBSession.add(group)
db_session.add(group)
def reduce_member_count(gid):
q = DBSession.query(Group).filter_by(id=gid)
def reduce_member_count(db_session, gid):
q = db_session.query(Group).filter_by(id=gid)
group = q.first()
group.member_count -= 1
DBSession.add(group)
db_session.add(group)
def insert(request, values):
......@@ -222,12 +229,12 @@ def insert(request, values):
user.user_name = values['user_name'].lower()
user.security_code_date = create_now()
remain = regenerate_security_code(user)
DBSession.add(user)
DBSession.flush()
for gid in values['groups']:
request.dbsession.add(user)
request.dbsession.flush()
for gid in values['groups']:
ug = UserGroup(user_id=user.id, group_id=gid)
DBSession.add(ug)
add_member_count(gid)
request.dbsession.add(ug)
add_member_count(request.dbsession, gid)
return user, remain
......@@ -248,7 +255,7 @@ def view_add(request):
except ValidationFailure:
resp['form'] = form.render()
return resp
user, remain = insert(request, dict(c.items()))
user, remain = insert(request, dict(c.items()))
send_email_security_code(
request, user, remain, 'Welcome new user', 'email-new-user',
'email-new-user.tpl')
......@@ -264,8 +271,8 @@ def view_add(request):
########
# Edit #
########
def user_group_set(user):
q = DBSession.query(UserGroup).filter_by(user_id=user.id)
def user_group_set(db_session, user):
q = db_session.query(UserGroup).filter_by(user_id=user.id)
r = []
for ug in q:
r.append(str(ug.group_id))
......@@ -278,27 +285,28 @@ def update(request, user, values):
user.user_name = values['user_name'].lower()
if user.id != request.user.id:
user.status = values['status']
DBSession.add(user)
existing = user_group_set(user)
request.dbsession.add(user)
existing = user_group_set(request.dbsession, user)
unused = existing - values['groups']
if unused:
q = DBSession.query(UserGroup).filter_by(user_id=user.id).filter(
UserGroup.group_id.in_(unused))
q = request.dbsession.query(UserGroup).\
filter_by(user_id=user.id).\
filter(UserGroup.group_id.in_(unused))
q.delete(synchronize_session=False)
for gid in unused:
reduce_member_count(gid)
reduce_member_count(request.dbsession, gid)
new = values['groups'] - existing
for gid in new:
for gid in new:
ug = UserGroup(user_id=user.id, group_id=gid)
DBSession.add(ug)
add_member_count(gid)
request.dbsession.add(ug)
add_member_count(request.dbsession, gid)
@view_config(
route_name='user-edit', renderer='templates/user/edit.pt',
permission='user-edit')
def view_edit(request):
q = DBSession.query(User).filter_by(id=request.matchdict['id'])
q = request.dbsession.query(User).filter_by(id=request.matchdict['id'])
user = q.first()
if not user:
return HTTPNotFound()
......@@ -309,7 +317,7 @@ def view_edit(request):
resp = dict(title=_('Edit user'))
if not request.POST:
d = user.to_dict()
d['groups'] = user_group_set(user)
d['groups'] = user_group_set(request.dbsession, user)
resp['form'] = form.render(appstruct=d)
return resp
if 'save' not in request.POST:
......@@ -329,12 +337,13 @@ def view_edit(request):
##########
# Delete #
##########
##########
@view_config(
route_name='user-delete', renderer='templates/user/delete.pt',
permission='user-edit')
def view_delete(request):
q = DBSession.query(User).filter_by(id=request.matchdict['id'])
q = request.dbsession.query(User).filter_by(id=request.matchdict['id'])
user = q.first()
if not user:
return HTTPNotFound()
......@@ -346,7 +355,7 @@ def view_delete(request):
return dict(title=_('Delete user'), user=user, form=form.render())
if 'delete' not in request.POST:
return HTTPFound(location=request.route_url('user'))
gid_list = user_group_set(user)
gid_list = user_group_set(request.dbsession, user)
for gid in gid_list:
reduce_member_count(gid)
data = dict(uid=user.id, email=user.email)
......
setup.py
......@@ -18,9 +18,11 @@ requires = [
'pyramid',
'pyramid_chameleon',
'pyramid_debugtoolbar',
'pyramid_retry',
'pyramid_tm',
'waitress',
'zope.sqlalchemy',
'transaction',
'psycopg2-binary',
'pytz',
'ziggurat-foundations',
......@@ -31,10 +33,10 @@ requires = [
'pyramid_mailer',
'requests',
'pyramid_rpc',
'opensipkd-base @ git+https://git.opensipkd.com/sugiana/opensipkd-base.git',
'opensipkd-iso8583 @ git+https://git.opensipkd.com/sugiana/opensipkd-iso8583.git',
'opensipkd-hitung @ git+https://git.opensipkd.com/sugiana/opensipkd-hitung.git',
'pyramid-linkaja @ git+https://git.opensipkd.com/sugiana/pyramid-linkaja.git',
'opensipkd-iso8583 @ '
'git+https://git.opensipkd.com/sugiana/opensipkd-iso8583.git',
'opensipkd-hitung @ '
'git+https://git.opensipkd.com/sugiana/opensipkd-hitung.git',
]
......@@ -72,10 +74,10 @@ setup(
'main = iso8583_web:main',
],
'console_scripts': [
'iso8583 = iso8583_web.scripts.forwarder:main',
'iso8583_web_client = iso8583_web.scripts.web_client:main',
'iso8583_web_client_linkaja = iso8583_web.scripts.web_client_linkaja:main',
'initialize_iso8583_web_db = iso8583_web.scripts.initialize_db:main',
'iso8583 = iso8583_web.iso8583:main',
'initialize_iso8583_web_db = '
'iso8583_web.scripts.initialize_db:main',
'iso8583_set_conf = iso8583_web.scripts.set_conf:main',
]
},
)
Markdown is supported
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!