 | Not Current Content This page was created during the initial stages of thinking about the kernel, however, it is not a complete representation of what ended up in the Kernel 1.0 RC-1 that Ian Boston worked on, and is retained here for historical purposes for the time being |
Information
This page documents the proposed plans and suggestions for splitting the kernel release from the main release and operating them on separate schedules.
- Information
- Summary
- High level terms
- What is a kernel release?
- What would be in a kernel distribution?
- What would NOT be in the kernel? (i.e. what does not belong)
- What about versioning, patches, and bug fixes?
- Kernel (proposed)
- Sakai Framework
- What about some standards for the kernel?
- What are the first steps?
- What about a roadmap?
- How does the cafe distribution fit into this?
- What about the view technology packages?
Summary
- Current issues that this addresses
- No clear direction for what the core services (kernel) are or what they accomplish
- Unclear what the core services in Sakai are and what they should actually be doing for the tool developers
- Critical framework code is untested and therefore unreliable and is often changed without evaluating the impact of the changes
- Tool development which depends on the core services is hard (for many reasons) and updates to services have to be rolled into the current release, even if they are not tested, because it is currently all one massive web
- Goals
- Separate the kernel services so they can be tested independently
- Create standards for the kernel services which ensure reliability and performance
- Allow the kernel to evolve independently from the tools and non-core services
- Make it easier for developers to rely on and link to the core services
- Benefits
- Higher quality and reliability for Sakai services (and therefore the product as a whole)
- Independent versioning of the kernel which allows the most stable current version to be used when an overall release occurs
- Better separation of dependencies which allows for more flexibility in modifying and replacing services when new technologies are available or improvements need to be made
- A well defined community process for working within the most critical parts of the codebase
- Easier tool development and dependency management for tools
High level terms
The Sakai kernel is a collection of core service that defined a consistent set of tool needs. The Sakai framework is a larger collection of software that supports tool development. The framework includes one or more portals, presentation technologies, and minimal administration tools.
What is a kernel release?
- The kernel is the collection of key services that Sakai tools can rely on. They are a set of published APIs that define a contract for services offered. The kernel does not have UI dependences, but the framework may. The kernel satisfies tool needs.
What would be in a kernel distribution?
The Sakai kernel should include:
-
- User management (authentication)
- Group management (by this I mean any group of users so sites, groups, sections, etc.)
- Central authorization/permissions management
- Content management
- Portalized views and navigations
- Service/Component management and configuration
- Tool management and support
- Entity management
- Event management
- Messaging/Notification support
- Centralized caching and clustering support
- Database management support
- Scheduling/Delayed activation support
- User data aggregation/filtering/searching support (this ties into tool support and other areas)
- Testing (unit, integration, system, load) support
- The kernel release would adhere to a set of well-defined best practices
- The kernel may eventually also include the following services and functionality
- Hierarchy service to allow hierarchical organization of various data
- Centralized authentication using something like CAS
What would NOT be in the kernel? (i.e. what does not belong)
- Tools and webapps are not part of the kernel, these are certainly what makes Sakai for the user but they are not the kernel
- Web/View technologies do not belong in the kernel unless they are specifically supporting the portal (framework, not kernel)
- Services which specifically support a single tool/function do not belong in the kernel, they should be packaged with that tool
- Services which support multiple tools but are not part of the core kernel mission should be placed into their own service package but not included in the kernel
What about versioning, patches, and bug fixes?
- The kernel will have a version which is independent from the rest of Sakai
- They kernel jars will be downloadable from the Sakai maven 2 repository
- This is in contrast to the rest of Sakai which should be downloaded as a package or built from source
- Eclipse would no longer have dependencies on other projects but will have all dependencies on the kernel managed as libraries
- The kernel would be available as source like always
- Versioning and bug fixes for the kernel release should be handled like a well organized large project release (e.g. Spring, Hibernate)
- RC jars - We would have Release Candidate (RC) sets of jars created as major changes and bug fixes go in leading up to the actual release. Something like 1.0-RC1 and 1.0-RC2. These would go into the main maven 2 repository. QA would test against RC jars and the final RC should be identical to the release.
- SNAPSHOT jars - I would also like to see the SNAPSHOT jars available in a Sakai maven 2 dev repository. These would be built daily and deployed to the maven2 dev repository automatically. This would allow developers to be working with a version of the kernel that is not even to an RC stage yet or is between RCs
- Release jars - The set of jars for a major release would be created once QA has completed testing. This would be similar to creating a set of minor release and bug fix jars
- Minor Release - Between major releases we could have .1 releases for new functionality (also known as Functionality updates)
- Bug Fixes - Between major releases and minor releases we could have .0.1 releases to put in patches to bugs (as needed)
- API compatibility convention
- We would want to follow the API compatibility convention I have seen other large projects follow. APIs would be guaranteed for major version numbers (i.e. 1.0 and 1.1 and 1.1.1 would be compatible via APIs, there might be API additions or even major implementation changes but compatibility is guaranteed)
- Higher versions (2.0) might still be compatible with lower versions (1.0) but that would not be guaranteed
- We would strive to document changes to APIs from major release to major release
- We would deprecate APIs which are going to be removed from the kernel for a complete major release cycle whenever possible
Kernel (proposed)
- The proposed kernel projects (based on the definition above)
archive (partial)
authz (partial)
cluster
component
content (partial)
course-management (api only)
db
email
entity
entitybroker
event
jobscheduler (partial)
master
message (partial)
memory (partial)
portal (partial)
reference (partial)
site (partial)
site-manage (partial)
test-harness
tool (partial)
user (partial)
util - Proof of concept here: https://source.sakaiproject.org/contrib/caret/kernel/
- Total build time is 2:30 on an older machine
Sakai Framework
- The current minimum packages that Sakai framework can startup with include:
access
alias
announcement
archive (partial)
authz
cluster
component
content (partial)
courier
course-management (api)
db
email
entity
event
jobscheduler
jsf
login
master
message
memory
portal
presence (api)
reference
site
site-manage
test-harness
tool
user (partial)
util
velocity- Some of these may not appear to fit the definition above but because of the current dependency graph cannot be removed, those would include:
- access
- alias
- announcement
- courier
- jsf
- login
- presence
- velocity
- Some of these may not appear to fit the definition above but because of the current dependency graph cannot be removed, those would include:
What about some standards for the kernel?
The kernel code standards would be very high. All code going into the kernel or changing in the kernel would need to be checked and all kernel code would be stress tested. Specific standards would need to be defined. All kernel code would need to have unit/integration tests written.
What are the first steps?
- The short term (2.6) steps are outlined here:
- Fix dependencies in the current code to allow allow the kernel to be as defined and remove the non-kernel projects
- Remove the usage of covers in all kernel code and replace it with setter injection (this enables better testing)
- Remove all covers and replace them with one master Sakai cover (this would support things like jruby, jython, and webservices)
- Interfaces in the kernel should be refactored to remove unneeded dependencies
- Add appropriate programmatic testing
- Create the first kernel distribution
What about a roadmap?
We are working on a roadmap to define the goals of the kernel release and distribution and to make sure we are working towards those goals and not just randomly mucking around in the kernel. The short term goals as outlined are the first step in this process.
How does the cafe distribution fit into this?
- What I currently have been able to trim the trunk cafe down to is the minimum packages that Sakai will actually build and startup with plus some additional packages to support developers
- Added projects
- entitybroker
- help
- Added projects
- Cafe trunk is here: https://source.sakaiproject.org/svn/cafe/trunk/
- Here is the current list of projects:
access
alias
announcement
archive (partial)
authz
cluster
component
content (partial)
courier
course-management (api)
db
email
entity
entitybroker
event
help
jobscheduler
jsf
login
master
message
memory
portal
presence (api)
reference
site
site-manage
test-harness
tool
user (partial)
util
velocity- I would like to see the following NOT included in cafe trunk (but this is currently not possible because of package interdependencies):
announcement
presence
courier
- I would like to see the following NOT included in cafe trunk (but this is currently not possible because of package interdependencies):
What about the view technology packages?
The view technology support in Sakai is important because of the way the portal handles servlets and therefore is critical to the operation of Sakai. We would certainly include this in the cafe distribution and main distribtution because these support developers and the tools they write.
We want to see the various view tech pieces shifted over into a common project like so and see dependencies from these packages reduced as much as possible.
- viewtech/
- /jsf
- /rsf
- /jsp
- /velocity
Comments (10)
Sep 23, 2007
Mark J. Norton says:
How does this kernel proposal different from what is now called the framework se...How does this kernel proposal different from what is now called the framework services? Is this just a terminology change, or would there will be a framework?
Sep 24, 2007
Aaron Zeckoski says:
The current framework services are not very well defined (in my opinion) nor are...The current framework services are not very well defined (in my opinion) nor are they available as a separate release from the main body of Sakai (as defined by the tools). I think this should be preceived as a clarification or defining of what the kernel is. It is also bringing together the best ideas from around the community so that we can hopefully define a roadmap for the kernel release.
Sep 26, 2007
Mark J. Norton says:
Well, 1 for that, Aaron. The current organization makes little or no sense to me...Well, +1 for that, Aaron. The current organization makes little or no sense to me. In my training programs, I describe the framework services as roughly divided into to "classes" of objects: run time services, and persistent services. Examples of run time services include the Tool Manager, Session Manager, etc. Persistent services include User, Site, AuthZGroup, etc.
That said, what services in the current collection labeled "framework services" would you move out? Granted, we could reduce inter-dependencies which would make it possible to leave out certain service (such as presence), but most of them seem to be framework-ish to me. I would, in fact, expand the collection to include Content Hosting.
Sep 26, 2007
Aaron Zeckoski says:
The list is above, and content hosting is included as it matches with one of the...The list is above, and content hosting is included as it matches with one of the key functions of the Sakai kernel (content management). The idea of runtime vs. persistent is interesting but since even the session manager actually does persist data I would say the majority of services do in fact have some persistence (except perhaps entity broker).
I mentioned some services (like persence) that I would remove but basically anything that does not meet the list at the top of things in a kernel release.
Nov 05, 2007
Mark J. Norton says:
As I mentioned in the Best Practices page, I don't think that UI support should ...As I mentioned in the Best Practices page, I don't think that UI support should be included in the kernel distribution. That would allow us to remove the JSF, Velocity, and RSF modules from the list above. I don't think the portal should be included in the kernel as it is an aggregator of UI data, rather than being the framework that Sakai is based on. A portal is NOT a requirement to create a meaningful collection of kernel service.
Note that by removing UI aspects from the kernel, it means that the Cafe distro would still have meaning. Cafe would add back in presentation support, the portal(s), admin tools, etc.
The kernel is not about tools. It is about providing shared, common services that can be leveraged to produce tools.
PS. This suggestion also implies changes to the way code is organized in Sakai, as well. Many of the awkward dependencies that come up in stripping down Sakai have to do with the fact that services cannot be easily separated out from the tools that use them (ie, User Management vs. the User service).
Nov 05, 2007
Ian Boston says:
I agree that we need to remove the UI technologies as far as possible from the K...I agree that we need to remove the UI technologies as far as possible from the Kernel, however one of the things that we need to do with the kernel is to make certain that it scales and performs. Thats not just limited to the component code. The portal is the bit of code that takes the most load, and like content hosting, if that breaks and doesnt perform then the kernel isnt worth using.
The second aim of the kernel is to provide an environment which a tool developer can create a tool within. Without a portal this cannot be done.
So our aim should be to minimise the number of UI components of the portal so that we dont drag in lots of dependencies. We should not have JSF, we should not have anything UI related in shared and we should try and use UI stacks for the kernel tools that are as simple and as light weight as possible. Most of all anything that we do in the kernel should not force a tool to adopt a view technology. IMHO the VelocityServlet and its heavy lifecycle comes close to this. I would prefer only to have very very light weight stacks in the portal. I think the current SkinnableCharonPortal has this approach right, where we only require, servlet, simple beans and a template render technology. I chose velocity, to be agnostic, but it could also be the IKAT part of RSF.
Nov 06, 2007
Mark J. Norton says:
> The portal is the bit of code that takes the most load, and like content hosti...> The portal is the bit of code that takes the most load, and like content hosting, if that breaks and doesn't perform then the kernel isn't worth using.
I would agree with this but for one thing: you are assuming that folks will always use one of the Sakai portals. We are well on our way to making Sakai tools run in other environments, like uPortal. Further, people may choose to develop their own portals from scratch. From that perspective, the kernel may well be worth using, even if Charon (etc) doesn't perform well.
Taking this a bit further, there are aspects of the portal which should be considered part of the kernel: request processing, session management, tool placement and management, sites, etc. Isn't this where the heavy lifting of portal related activities occur? Perhaps the portal needs a bit more refactoring to clearly separate presentation and UI.
> The second aim of the kernel is to provide an environment which a tool developer can create a tool within.
As I see it, the kernel is a platform to built tools ON, not within. Yeah, maybe it's just semantics, but my point is that you are again assuming that tools are only reason for the existence of the kernel services. Isn't it the case that people often use Sakai services via web service interfaces?
To me, the kernel is a collection of abstract services that provide capabilities that can be used to develop teaching, learning, and collaboration systems. Sakai's abstract architecture is layered for this very reason. Each layer makes no assumptions about what it is being used for at higher levels. Practical considerations may impose some limitations and restrictions, but the goal is to have a flexible architecture that can be widely used.
-------------
All that said, I can see that my view is somewhat extreme. If the kernel distribution was pared down to only abstract services (APIs plus their implementations), it isn't very useful (as is). It would be nice to be able to run tests including harnesses that have a UI (not just JUnit). We might also want some kind of developer tools.
As such, I'd propose that we don't include the Charon portal (or it's variants), but instead include Mercury or some enhanced form of it. To my knowledge, Mercury has no presentation technology requirements at all, being purely Servlet based.
Nov 06, 2007
Steve Lay says:
Extreme seems good to me. The Linux kernel is pretty useful and it doesn't have...Extreme seems good to me.
The Linux kernel is pretty useful and it doesn't have a UI. I presume that this is the model that people are thinking about with a framework/tool (aka kernel/package) distribution model.
Running tests that have a UI - much harder than running tests against code which is designed to be used by only other code. It will be much easier to get quality testing of the kernel up to scratch if there is no UI to fuss over. (UI brings in locale issues too remember!)
Also, if we have a kernel/package distribution model the type of management process required at the two levels is different (cf Ubuntu or Red Hat against the Linux kernel itself).
As a thought experiment, how would you all feel if somebody threw away all the tools/UI code and just used the Sakai kernel as a platform for their own system? Would you support them? Would you feel that they were operating outside the spirit of what this community feels comfortable with?
Oct 03, 2007
Mustansar Mehmood says:
I am not sure if this is some thing that goes here very directly but since separ...I am not sure if this is some thing that goes here very directly but since separating the kernel from the applications is being discussed and in my opinion a very good start to make it a framework +1 for that. Comment/suggestion I want to make here is that it will offer a huge support to sakai from commercial and non commercial groups if it was possible to so run sakai on other J2EE compliant application servers like Geronimo, Weblogic, Jboss Websphere , Glass Fish and so on . I understand that there are components like Web-Dav etc that are tomcat specific to some extent. But for what it is worth, here it is.
Nov 13, 2007
Ray Davis says:
Even if there are no current dependencies that force it to be there, I'd like to...Even if there are no current dependencies that force it to be there, I'd like to see "entitybroker" included so as to improve our chances to get cleaner dependency trees going forward.